2023-08-11T15:15:38.977 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-11T15:15:38.978 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050 --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/orig.config.yaml 2023-08-11T15:15:39.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T15:15:39.119 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050 branch: wip-rishabh-2023Aug1-b4 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7366050' kernel: client: branch: testing flavor: default kdb: true sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Aug1-b4 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Aug1-b4 sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Aug1-b4 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi192.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 7fda95956ac10132c9b74016ba832db907df09fa timestamp: 2023-08-10_20:16:46 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-11T15:15:39.120 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa; will attempt to use it 2023-08-11T15:15:39.121 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks 2023-08-11T15:15:39.121 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-11T15:15:39.123 INFO:teuthology.task.internal:Checking packages... 2023-08-11T15:15:39.151 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '92684807f016ce045ac66e37fdde6f8c47857bcc' 2023-08-11T15:15:39.151 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-11T15:15:39.152 INFO:teuthology.packaging:ref: None 2023-08-11T15:15:39.152 INFO:teuthology.packaging:tag: None 2023-08-11T15:15:39.152 INFO:teuthology.packaging:branch: wip-rishabh-2023Aug1-b4 2023-08-11T15:15:39.152 INFO:teuthology.packaging:sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:15:39.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Aug1-b4 2023-08-11T15:15:39.371 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5437.g92684807 2023-08-11T15:15:39.371 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-11T15:15:39.378 INFO:teuthology.task.internal:no buildpackages task found 2023-08-11T15:15:39.378 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-11T15:15:39.400 INFO:teuthology.task.internal:Saving configuration 2023-08-11T15:15:39.421 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-11T15:15:39.428 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T15:15:39.453 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:10:31.166259', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-11T15:15:39.478 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:10:31.164699', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-11T15:15:39.497 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:10:31.162949', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-11T15:15:39.498 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-11T15:15:39.509 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-08-11T15:15:39.509 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-08-11T15:15:39.509 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-08-11T15:15:39.509 INFO:teuthology.run_tasks:Running task console_log... 2023-08-11T15:15:39.681 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faa413c20d0>, signals=[15]) 2023-08-11T15:15:39.682 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-11T15:15:39.690 INFO:teuthology.task.internal:Opening connections... 2023-08-11T15:15:39.691 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:15:39.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:15:39.760 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2023-08-11T15:15:39.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:15:39.829 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2023-08-11T15:15:39.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:15:39.897 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-11T15:15:39.903 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-08-11T15:15:39.924 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-08-11T15:15:39.925 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-08-11T15:15:39.985 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-08-11T15:15:39.986 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.6 (Ootpa)" 2023-08-11T15:15:39.986 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.6" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-08-11T15:15:39.987 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-11T15:15:39.988 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T15:15:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:15:39.988 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T15:15:39.988 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-11T15:15:39.988 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-11T15:15:39.988 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-11T15:15:39.989 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-08-11T15:15:40.013 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2023-08-11T15:15:40.034 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2023-08-11T15:15:40.035 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2023-08-11T15:15:40.094 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Red Hat Enterprise Linux" 2023-08-11T15:15:40.094 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8.6 (Ootpa)" 2023-08-11T15:15:40.094 INFO:teuthology.orchestra.run.smithi134.stdout:ID="rhel" 2023-08-11T15:15:40.095 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="fedora" 2023-08-11T15:15:40.095 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8.6" 2023-08-11T15:15:40.095 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8" 2023-08-11T15:15:40.095 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-11T15:15:40.095 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2023-08-11T15:15:40.095 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-11T15:15:40.095 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.redhat.com/" 2023-08-11T15:15:40.096 INFO:teuthology.orchestra.run.smithi134.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-11T15:15:40.096 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T15:15:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:15:40.096 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T15:15:40.096 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-11T15:15:40.096 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-11T15:15:40.096 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-11T15:15:40.097 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2023-08-11T15:15:40.113 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2023-08-11T15:15:40.133 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2023-08-11T15:15:40.134 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2023-08-11T15:15:40.193 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Red Hat Enterprise Linux" 2023-08-11T15:15:40.193 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="8.6 (Ootpa)" 2023-08-11T15:15:40.193 INFO:teuthology.orchestra.run.smithi192.stdout:ID="rhel" 2023-08-11T15:15:40.193 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="fedora" 2023-08-11T15:15:40.194 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="8.6" 2023-08-11T15:15:40.194 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el8" 2023-08-11T15:15:40.194 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-11T15:15:40.194 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2023-08-11T15:15:40.194 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-11T15:15:40.194 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.redhat.com/" 2023-08-11T15:15:40.194 INFO:teuthology.orchestra.run.smithi192.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-11T15:15:40.195 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T15:15:40.195 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:15:40.195 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T15:15:40.195 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-11T15:15:40.195 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-11T15:15:40.195 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-11T15:15:40.195 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2023-08-11T15:15:40.213 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-11T15:15:40.223 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-11T15:15:40.231 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-11T15:15:40.231 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-08-11T15:15:40.234 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2023-08-11T15:15:40.236 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2023-08-11T15:15:40.253 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-11T15:15:40.264 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-11T15:15:40.265 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-08-11T15:15:40.293 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2023-08-11T15:15:40.295 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2023-08-11T15:15:40.336 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-11T15:15:40.408 INFO:teuthology.run_tasks:Running task kernel... 2023-08-11T15:15:40.424 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'} 2023-08-11T15:15:40.424 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-08-11T15:15:40.425 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-08-11T15:15:40.425 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T15:15:40.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:15:40.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-08-11T15:15:40.628 DEBUG:teuthology.task.kernel:sha1 for client.0 is a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:15:40.628 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a7fb1265323db972dd333f71b9a53e9479f62e37"... 2023-08-11T15:15:40.628 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-08-11T15:15:40.649 INFO:teuthology.orchestra.run.smithi032.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-11T15:15:40.649 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:15:40.649 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-11T15:15:40.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum upgrade -y linux-firmware 2023-08-11T15:15:41.880 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-08-11T15:15:41.880 INFO:teuthology.orchestra.run.smithi032.stdout:Unable to read consumer identity 2023-08-11T15:15:43.247 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-08-11T15:15:44.015 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-08-11T15:15:47.690 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 411 kB/s | 24 kB 00:00 2023-08-11T15:15:48.475 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-08-11T15:15:48.476 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2023-08-11T15:15:48.477 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-08-11T15:15:48.613 INFO:teuthology.task.kernel:Downloading kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on client.0... 2023-08-11T15:15:48.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T15:15:48.752 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/8/flavors/default/x86_64/ 2023-08-11T15:15:48.752 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/8/flavors/default/x86_64/ --input-file=- 2023-08-11T15:15:48.755 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi032.front.sepia.ceph.com to complete... 2023-08-11T15:16:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11 15:16:04 URL:https://1.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/8/flavors/default/x86_64/kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm [380807140/380807140] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-11T15:16:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:FINISHED --2023-08-11 15:16:04-- 2023-08-11T15:16:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:Total wall clock time: 16s 2023-08-11T15:16:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:Downloaded: 1 files, 363M in 16s (23.0 MB/s) 2023-08-11T15:16:04.788 INFO:teuthology.task.kernel:Installing kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on client.0... 2023-08-11T15:16:04.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-11T15:16:06.452 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying... ######################################## 2023-08-11T15:16:06.514 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing... ######################################## 2023-08-11T15:16:06.600 INFO:teuthology.orchestra.run.smithi032.stdout:Updating / installing... 2023-08-11T15:16:47.311 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-6.5.0_rc4_ga7fb1265323d-1 ######################################## 2023-08-11T15:17:16.821 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-11T15:17:16.821 DEBUG:teuthology.orchestra.run.smithi032:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-11T15:17:16.874 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:16.875 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/config-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:16.875 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/vmlinuz-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:16.875 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:16.876 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel 2023-08-11T15:17:16.876 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch 2023-08-11T15:17:16.876 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86 2023-08-11T15:17:16.876 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto 2023-08-11T15:17:16.876 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-11T15:17:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-11T15:17:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-11T15:17:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-11T15:17:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-11T15:17:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-11T15:17:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-11T15:17:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-11T15:17:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-11T15:17:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-11T15:17:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-11T15:17:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-11T15:17:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-11T15:17:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-11T15:17:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-11T15:17:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-11T15:17:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-11T15:17:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-11T15:17:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-11T15:17:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-11T15:17:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel 2023-08-11T15:17:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu 2023-08-11T15:17:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce 2023-08-11T15:17:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-11T15:17:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm 2023-08-11T15:17:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-11T15:17:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-11T15:17:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm.ko 2023-08-11T15:17:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto 2023-08-11T15:17:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ansi_cprng.ko 2023-08-11T15:17:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/anubis.ko 2023-08-11T15:17:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/arc4.ko 2023-08-11T15:17:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx 2023-08-11T15:17:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_memcpy.ko 2023-08-11T15:17:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_pq.ko 2023-08-11T15:17:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-11T15:17:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_tx.ko 2023-08-11T15:17:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_xor.ko 2023-08-11T15:17:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/raid6test.ko 2023-08-11T15:17:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authenc.ko 2023-08-11T15:17:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authencesn.ko 2023-08-11T15:17:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blake2b_generic.ko 2023-08-11T15:17:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_common.ko 2023-08-11T15:17:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_generic.ko 2023-08-11T15:17:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/camellia_generic.ko 2023-08-11T15:17:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast5_generic.ko 2023-08-11T15:17:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast6_generic.ko 2023-08-11T15:17:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast_common.ko 2023-08-11T15:17:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ccm.ko 2023-08-11T15:17:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cmac.ko 2023-08-11T15:17:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crc32_generic.ko 2023-08-11T15:17:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_engine.ko 2023-08-11T15:17:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_user.ko 2023-08-11T15:17:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cts.ko 2023-08-11T15:17:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/des_generic.ko 2023-08-11T15:17:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/dh_generic.ko 2023-08-11T15:17:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/echainiv.ko 2023-08-11T15:17:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/essiv.ko 2023-08-11T15:17:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/fcrypt.ko 2023-08-11T15:17:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/khazad.ko 2023-08-11T15:17:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4.ko 2023-08-11T15:17:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4hc.ko 2023-08-11T15:17:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/md4.ko 2023-08-11T15:17:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/michael_mic.ko 2023-08-11T15:17:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcbc.ko 2023-08-11T15:17:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcrypt.ko 2023-08-11T15:17:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/rmd160.ko 2023-08-11T15:17:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/seed.ko 2023-08-11T15:17:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/serpent_generic.ko 2023-08-11T15:17:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tcrypt.ko 2023-08-11T15:17:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tea.ko 2023-08-11T15:17:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_common.ko 2023-08-11T15:17:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_generic.ko 2023-08-11T15:17:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/vmac.ko 2023-08-11T15:17:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/wp512.ko 2023-08-11T15:17:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xcbc.ko 2023-08-11T15:17:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xor.ko 2023-08-11T15:17:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xxhash_generic.ko 2023-08-11T15:17:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers 2023-08-11T15:17:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi 2023-08-11T15:17:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-11T15:17:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_pad.ko 2023-08-11T15:17:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/custom_method.ko 2023-08-11T15:17:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/ec_sys.ko 2023-08-11T15:17:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/platform_profile.ko 2023-08-11T15:17:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbs.ko 2023-08-11T15:17:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbshc.ko 2023-08-11T15:17:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/video.ko 2023-08-11T15:17:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata 2023-08-11T15:17:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/acard-ahci.ko 2023-08-11T15:17:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ahci_platform.ko 2023-08-11T15:17:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ata_generic.ko 2023-08-11T15:17:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/libahci_platform.ko 2023-08-11T15:17:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_acpi.ko 2023-08-11T15:17:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ali.ko 2023-08-11T15:17:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_amd.ko 2023-08-11T15:17:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_artop.ko 2023-08-11T15:17:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atiixp.ko 2023-08-11T15:17:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atp867x.ko 2023-08-11T15:17:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd640.ko 2023-08-11T15:17:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd64x.ko 2023-08-11T15:17:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cypress.ko 2023-08-11T15:17:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_efar.ko 2023-08-11T15:17:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt366.ko 2023-08-11T15:17:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt37x.ko 2023-08-11T15:17:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-11T15:17:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-11T15:17:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it8213.ko 2023-08-11T15:17:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it821x.ko 2023-08-11T15:17:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_jmicron.ko 2023-08-11T15:17:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_marvell.ko 2023-08-11T15:17:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_mpiix.ko 2023-08-11T15:17:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_netcell.ko 2023-08-11T15:17:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ninja32.ko 2023-08-11T15:17:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87410.ko 2023-08-11T15:17:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87415.ko 2023-08-11T15:17:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_oldpiix.ko 2023-08-11T15:17:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_opti.ko 2023-08-11T15:17:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_optidma.ko 2023-08-11T15:17:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pcmcia.ko 2023-08-11T15:17:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-11T15:17:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-11T15:17:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_piccolo.ko 2023-08-11T15:17:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_rdc.ko 2023-08-11T15:17:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sch.ko 2023-08-11T15:17:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_serverworks.ko 2023-08-11T15:17:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sil680.ko 2023-08-11T15:17:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sis.ko 2023-08-11T15:17:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sl82c105.ko 2023-08-11T15:17:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_triflex.ko 2023-08-11T15:17:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_via.ko 2023-08-11T15:17:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pdc_adma.ko 2023-08-11T15:17:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_inic162x.ko 2023-08-11T15:17:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_mv.ko 2023-08-11T15:17:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_nv.ko 2023-08-11T15:17:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_promise.ko 2023-08-11T15:17:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_qstor.ko 2023-08-11T15:17:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil.ko 2023-08-11T15:17:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil24.ko 2023-08-11T15:17:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sis.ko 2023-08-11T15:17:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_svw.ko 2023-08-11T15:17:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sx4.ko 2023-08-11T15:17:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_uli.ko 2023-08-11T15:17:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_via.ko 2023-08-11T15:17:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_vsc.ko 2023-08-11T15:17:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay 2023-08-11T15:17:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay/ks0108.ko 2023-08-11T15:17:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base 2023-08-11T15:17:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap 2023-08-11T15:17:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-11T15:17:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-11T15:17:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma 2023-08-11T15:17:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma/bcma.ko 2023-08-11T15:17:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block 2023-08-11T15:17:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe 2023-08-11T15:17:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe/aoe.ko 2023-08-11T15:17:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/brd.ko 2023-08-11T15:17:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd 2023-08-11T15:17:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd/drbd.ko 2023-08-11T15:17:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/floppy.ko 2023-08-11T15:17:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/loop.ko 2023-08-11T15:17:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx 2023-08-11T15:17:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-11T15:17:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/nbd.ko 2023-08-11T15:17:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk 2023-08-11T15:17:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk/null_blk.ko 2023-08-11T15:17:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/pktcdvd.ko 2023-08-11T15:17:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/rbd.ko 2023-08-11T15:17:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/virtio_blk.ko 2023-08-11T15:17:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback 2023-08-11T15:17:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-11T15:17:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkfront.ko 2023-08-11T15:17:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram 2023-08-11T15:17:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram/zram.ko 2023-08-11T15:17:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char 2023-08-11T15:17:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hangcheck-timer.ko 2023-08-11T15:17:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random 2023-08-11T15:17:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-11T15:17:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-11T15:17:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-11T15:17:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/via-rng.ko 2023-08-11T15:17:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-11T15:17:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi 2023-08-11T15:17:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-11T15:17:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-11T15:17:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-11T15:17:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-11T15:17:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-11T15:17:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-11T15:17:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/lp.ko 2023-08-11T15:17:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave 2023-08-11T15:17:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave/mwave.ko 2023-08-11T15:17:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ppdev.ko 2023-08-11T15:17:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tlclk.ko 2023-08-11T15:17:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm 2023-08-11T15:17:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm.ko 2023-08-11T15:17:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-11T15:17:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-11T15:17:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-11T15:17:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-11T15:17:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-11T15:17:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/uv_mmtimer.ko 2023-08-11T15:17:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/virtio_console.ko 2023-08-11T15:17:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq 2023-08-11T15:17:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-11T15:17:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-11T15:17:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-11T15:17:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-11T15:17:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-11T15:17:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-11T15:17:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto 2023-08-11T15:17:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp 2023-08-11T15:17:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-11T15:17:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp.ko 2023-08-11T15:17:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel 2023-08-11T15:17:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat 2023-08-11T15:17:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common 2023-08-11T15:17:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-11T15:17:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-11T15:17:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-11T15:17:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-aes.ko 2023-08-11T15:17:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-sha.ko 2023-08-11T15:17:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio 2023-08-11T15:17:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-11T15:17:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca 2023-08-11T15:17:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca/dca.ko 2023-08-11T15:17:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq 2023-08-11T15:17:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-11T15:17:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma 2023-08-11T15:17:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw 2023-08-11T15:17:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-11T15:17:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat 2023-08-11T15:17:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-11T15:17:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac 2023-08-11T15:17:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/amd64_edac.ko 2023-08-11T15:17:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/e752x_edac.ko 2023-08-11T15:17:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/edac_mce_amd.ko 2023-08-11T15:17:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3000_edac.ko 2023-08-11T15:17:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3200_edac.ko 2023-08-11T15:17:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5100_edac.ko 2023-08-11T15:17:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5400_edac.ko 2023-08-11T15:17:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7300_edac.ko 2023-08-11T15:17:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7core_edac.ko 2023-08-11T15:17:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i82975x_edac.ko 2023-08-11T15:17:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/ie31200_edac.ko 2023-08-11T15:17:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/sb_edac.ko 2023-08-11T15:17:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/x38_edac.ko 2023-08-11T15:17:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware 2023-08-11T15:17:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/edd.ko 2023-08-11T15:17:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-11T15:17:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio 2023-08-11T15:17:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-ich.ko 2023-08-11T15:17:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-11T15:17:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid 2023-08-11T15:17:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-appleir.ko 2023-08-11T15:17:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-aureal.ko 2023-08-11T15:17:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-axff.ko 2023-08-11T15:17:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-dr.ko 2023-08-11T15:17:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elecom.ko 2023-08-11T15:17:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elo.ko 2023-08-11T15:17:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-emsff.ko 2023-08-11T15:17:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gaff.ko 2023-08-11T15:17:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gt683r.ko 2023-08-11T15:17:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gyration.ko 2023-08-11T15:17:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-11T15:17:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-11T15:17:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtekff.ko 2023-08-11T15:17:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-hyperv.ko 2023-08-11T15:17:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-icade.ko 2023-08-11T15:17:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-keytouch.ko 2023-08-11T15:17:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-kye.ko 2023-08-11T15:17:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lcpower.ko 2023-08-11T15:17:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-led.ko 2023-08-11T15:17:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lenovo.ko 2023-08-11T15:17:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-11T15:17:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-11T15:17:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-multitouch.ko 2023-08-11T15:17:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-ortek.ko 2023-08-11T15:17:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-penmount.ko 2023-08-11T15:17:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-petalynx.ko 2023-08-11T15:17:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-picolcd.ko 2023-08-11T15:17:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-pl.ko 2023-08-11T15:17:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-plantronics.ko 2023-08-11T15:17:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-primax.ko 2023-08-11T15:17:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-rmi.ko 2023-08-11T15:17:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-11T15:17:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-common.ko 2023-08-11T15:17:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-11T15:17:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-11T15:17:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-11T15:17:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-11T15:17:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-11T15:17:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-11T15:17:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-11T15:17:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-11T15:17:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-11T15:17:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat.ko 2023-08-11T15:17:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-saitek.ko 2023-08-11T15:17:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-samsung.ko 2023-08-11T15:17:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-11T15:17:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sjoy.ko 2023-08-11T15:17:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sony.ko 2023-08-11T15:17:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-speedlink.ko 2023-08-11T15:17:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-steelseries.ko 2023-08-11T15:17:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sunplus.ko 2023-08-11T15:17:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-11T15:17:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tivo.ko 2023-08-11T15:17:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tmff.ko 2023-08-11T15:17:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-topseed.ko 2023-08-11T15:17:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-twinhan.ko 2023-08-11T15:17:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-uclogic.ko 2023-08-11T15:17:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-waltop.ko 2023-08-11T15:17:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-wiimote.ko 2023-08-11T15:17:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-xinmo.ko 2023-08-11T15:17:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zpff.ko 2023-08-11T15:17:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zydacron.ko 2023-08-11T15:17:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/uhid.ko 2023-08-11T15:17:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/wacom.ko 2023-08-11T15:17:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv 2023-08-11T15:17:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_balloon.ko 2023-08-11T15:17:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_utils.ko 2023-08-11T15:17:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_vmbus.ko 2023-08-11T15:17:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon 2023-08-11T15:17:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru.ko 2023-08-11T15:17:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru3.ko 2023-08-11T15:17:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-11T15:17:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7414.ko 2023-08-11T15:17:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7418.ko 2023-08-11T15:17:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adc128d818.ko 2023-08-11T15:17:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1025.ko 2023-08-11T15:17:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1026.ko 2023-08-11T15:17:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1029.ko 2023-08-11T15:17:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1031.ko 2023-08-11T15:17:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm9240.ko 2023-08-11T15:17:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ads7828.ko 2023-08-11T15:17:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7410.ko 2023-08-11T15:17:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7411.ko 2023-08-11T15:17:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7462.ko 2023-08-11T15:17:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7470.ko 2023-08-11T15:17:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7475.ko 2023-08-11T15:17:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7x10.ko 2023-08-11T15:17:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/amc6821.ko 2023-08-11T15:17:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/applesmc.ko 2023-08-11T15:17:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asb100.ko 2023-08-11T15:17:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asc7621.ko 2023-08-11T15:17:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-11T15:17:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/atxp1.ko 2023-08-11T15:17:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/coretemp.ko 2023-08-11T15:17:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-11T15:17:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dme1737.ko 2023-08-11T15:17:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds1621.ko 2023-08-11T15:17:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds620.ko 2023-08-11T15:17:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc1403.ko 2023-08-11T15:17:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc6w201.ko 2023-08-11T15:17:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71805f.ko 2023-08-11T15:17:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71882fg.ko 2023-08-11T15:17:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f75375s.ko 2023-08-11T15:17:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fam15h_power.ko 2023-08-11T15:17:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fschmd.ko 2023-08-11T15:17:16.948 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g760a.ko 2023-08-11T15:17:16.948 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g762.ko 2023-08-11T15:17:16.948 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl518sm.ko 2023-08-11T15:17:16.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl520sm.ko 2023-08-11T15:17:16.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-11T15:17:16.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5500_temp.ko 2023-08-11T15:17:16.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5k_amb.ko 2023-08-11T15:17:16.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmaem.ko 2023-08-11T15:17:16.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmpex.ko 2023-08-11T15:17:16.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina209.ko 2023-08-11T15:17:16.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina2xx.ko 2023-08-11T15:17:16.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/it87.ko 2023-08-11T15:17:16.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k10temp.ko 2023-08-11T15:17:16.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k8temp.ko 2023-08-11T15:17:16.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lineage-pem.ko 2023-08-11T15:17:16.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm63.ko 2023-08-11T15:17:16.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm73.ko 2023-08-11T15:17:16.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm75.ko 2023-08-11T15:17:16.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm77.ko 2023-08-11T15:17:16.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm78.ko 2023-08-11T15:17:16.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm80.ko 2023-08-11T15:17:16.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm83.ko 2023-08-11T15:17:16.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm85.ko 2023-08-11T15:17:16.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm87.ko 2023-08-11T15:17:16.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm90.ko 2023-08-11T15:17:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm92.ko 2023-08-11T15:17:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm93.ko 2023-08-11T15:17:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95234.ko 2023-08-11T15:17:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95241.ko 2023-08-11T15:17:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95245.ko 2023-08-11T15:17:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc2945.ko 2023-08-11T15:17:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4151.ko 2023-08-11T15:17:16.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4215.ko 2023-08-11T15:17:16.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4222.ko 2023-08-11T15:17:16.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4245.ko 2023-08-11T15:17:16.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4260.ko 2023-08-11T15:17:16.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4261.ko 2023-08-11T15:17:16.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max16065.ko 2023-08-11T15:17:16.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1619.ko 2023-08-11T15:17:16.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1668.ko 2023-08-11T15:17:16.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max197.ko 2023-08-11T15:17:16.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6639.ko 2023-08-11T15:17:16.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6650.ko 2023-08-11T15:17:16.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6697.ko 2023-08-11T15:17:16.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/mcp3021.ko 2023-08-11T15:17:16.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6683.ko 2023-08-11T15:17:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775-core.ko 2023-08-11T15:17:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775.ko 2023-08-11T15:17:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct7802.ko 2023-08-11T15:17:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87360.ko 2023-08-11T15:17:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87427.ko 2023-08-11T15:17:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pcf8591.ko 2023-08-11T15:17:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus 2023-08-11T15:17:16.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-11T15:17:16.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-11T15:17:16.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-11T15:17:16.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-11T15:17:16.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-11T15:17:16.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-11T15:17:16.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-11T15:17:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-11T15:17:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-11T15:17:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-11T15:17:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-11T15:17:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-11T15:17:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/powr1220.ko 2023-08-11T15:17:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5627.ko 2023-08-11T15:17:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5636.ko 2023-08-11T15:17:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-11T15:17:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht15.ko 2023-08-11T15:17:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht21.ko 2023-08-11T15:17:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/shtc1.ko 2023-08-11T15:17:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sis5595.ko 2023-08-11T15:17:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47b397.ko 2023-08-11T15:17:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m1.ko 2023-08-11T15:17:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m192.ko 2023-08-11T15:17:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/thmc50.ko 2023-08-11T15:17:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp102.ko 2023-08-11T15:17:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp103.ko 2023-08-11T15:17:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp401.ko 2023-08-11T15:17:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp421.ko 2023-08-11T15:17:16.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via-cputemp.ko 2023-08-11T15:17:16.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via686a.ko 2023-08-11T15:17:16.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt1211.ko 2023-08-11T15:17:16.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt8231.ko 2023-08-11T15:17:16.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627ehf.ko 2023-08-11T15:17:16.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627hf.ko 2023-08-11T15:17:16.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83781d.ko 2023-08-11T15:17:16.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83791d.ko 2023-08-11T15:17:16.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83792d.ko 2023-08-11T15:17:16.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83793.ko 2023-08-11T15:17:16.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83795.ko 2023-08-11T15:17:16.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l785ts.ko 2023-08-11T15:17:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l786ng.ko 2023-08-11T15:17:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c 2023-08-11T15:17:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos 2023-08-11T15:17:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-11T15:17:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-11T15:17:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses 2023-08-11T15:17:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-11T15:17:16.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-11T15:17:16.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-11T15:17:16.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-11T15:17:16.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-11T15:17:16.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-11T15:17:16.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-11T15:17:16.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-11T15:17:16.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-11T15:17:16.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-11T15:17:16.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-11T15:17:16.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-11T15:17:16.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-11T15:17:16.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-11T15:17:16.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-11T15:17:16.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-11T15:17:16.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-11T15:17:16.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-11T15:17:16.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-11T15:17:16.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-11T15:17:16.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-11T15:17:16.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-11T15:17:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-11T15:17:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-dev.ko 2023-08-11T15:17:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-mux.ko 2023-08-11T15:17:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-11T15:17:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-smbus.ko 2023-08-11T15:17:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-stub.ko 2023-08-11T15:17:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband 2023-08-11T15:17:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core 2023-08-11T15:17:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-11T15:17:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_core.ko 2023-08-11T15:17:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-11T15:17:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-11T15:17:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-11T15:17:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-11T15:17:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-11T15:17:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw 2023-08-11T15:17:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4 2023-08-11T15:17:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-11T15:17:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4 2023-08-11T15:17:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-11T15:17:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5 2023-08-11T15:17:16.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-11T15:17:16.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca 2023-08-11T15:17:16.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-11T15:17:16.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp 2023-08-11T15:17:16.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib 2023-08-11T15:17:16.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-11T15:17:16.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser 2023-08-11T15:17:16.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-11T15:17:16.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert 2023-08-11T15:17:16.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-11T15:17:16.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp 2023-08-11T15:17:16.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-11T15:17:16.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt 2023-08-11T15:17:16.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-11T15:17:16.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input 2023-08-11T15:17:16.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport 2023-08-11T15:17:16.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-11T15:17:16.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-11T15:17:16.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/gameport.ko 2023-08-11T15:17:16.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/lightning.ko 2023-08-11T15:17:16.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/ns558.ko 2023-08-11T15:17:16.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/joydev.ko 2023-08-11T15:17:16.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard 2023-08-11T15:17:16.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-11T15:17:16.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse 2023-08-11T15:17:16.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/appletouch.ko 2023-08-11T15:17:16.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/bcm5974.ko 2023-08-11T15:17:16.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/cyapatp.ko 2023-08-11T15:17:16.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-11T15:17:16.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/sermouse.ko 2023-08-11T15:17:16.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-11T15:17:16.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-11T15:17:16.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-11T15:17:16.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4 2023-08-11T15:17:16.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-11T15:17:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio 2023-08-11T15:17:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/altera_ps2.ko 2023-08-11T15:17:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/arc_ps2.ko 2023-08-11T15:17:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-11T15:17:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/serio_raw.ko 2023-08-11T15:17:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/sparse-keymap.ko 2023-08-11T15:17:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu 2023-08-11T15:17:16.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd 2023-08-11T15:17:16.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-11T15:17:16.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds 2023-08-11T15:17:16.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-blinkm.ko 2023-08-11T15:17:16.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lm3530.ko 2023-08-11T15:17:16.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lp3944.ko 2023-08-11T15:17:16.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-ss4200.ko 2023-08-11T15:17:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger 2023-08-11T15:17:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-11T15:17:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-11T15:17:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-11T15:17:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-11T15:17:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-11T15:17:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-11T15:17:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-11T15:17:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-11T15:17:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md 2023-08-11T15:17:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache 2023-08-11T15:17:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache/bcache.ko 2023-08-11T15:17:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-bio-prison.ko 2023-08-11T15:17:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache-smq.ko 2023-08-11T15:17:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache.ko 2023-08-11T15:17:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-crypt.ko 2023-08-11T15:17:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-delay.ko 2023-08-11T15:17:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-flakey.ko 2023-08-11T15:17:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-log-userspace.ko 2023-08-11T15:17:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-multipath.ko 2023-08-11T15:17:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-queue-length.ko 2023-08-11T15:17:16.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-raid.ko 2023-08-11T15:17:16.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-round-robin.ko 2023-08-11T15:17:16.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-service-time.ko 2023-08-11T15:17:16.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-switch.ko 2023-08-11T15:17:16.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-thin-pool.ko 2023-08-11T15:17:16.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-verity.ko 2023-08-11T15:17:16.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/faulty.ko 2023-08-11T15:17:16.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/linear.ko 2023-08-11T15:17:16.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/multipath.ko 2023-08-11T15:17:16.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data 2023-08-11T15:17:16.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-11T15:17:16.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid0.ko 2023-08-11T15:17:16.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid1.ko 2023-08-11T15:17:16.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid10.ko 2023-08-11T15:17:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid456.ko 2023-08-11T15:17:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message 2023-08-11T15:17:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion 2023-08-11T15:17:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptbase.ko 2023-08-11T15:17:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptctl.ko 2023-08-11T15:17:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptfc.ko 2023-08-11T15:17:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptlan.ko 2023-08-11T15:17:16.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptsas.ko 2023-08-11T15:17:16.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptscsih.ko 2023-08-11T15:17:16.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptspi.ko 2023-08-11T15:17:16.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd 2023-08-11T15:17:16.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_ich.ko 2023-08-11T15:17:16.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_sch.ko 2023-08-11T15:17:16.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/mfd-core.ko 2023-08-11T15:17:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/sm501.ko 2023-08-11T15:17:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/viperboard.ko 2023-08-11T15:17:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/vx855.ko 2023-08-11T15:17:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/wl1273-core.ko 2023-08-11T15:17:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc 2023-08-11T15:17:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl 2023-08-11T15:17:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-11T15:17:16.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds9802als.ko 2023-08-11T15:17:16.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds990x.ko 2023-08-11T15:17:16.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/bh1770glc.ko 2023-08-11T15:17:16.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710 2023-08-11T15:17:16.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710/cb710.ko 2023-08-11T15:17:16.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom 2023-08-11T15:17:16.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/at24.ko 2023-08-11T15:17:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-11T15:17:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-11T15:17:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/max6875.ko 2023-08-11T15:17:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/enclosure.ko 2023-08-11T15:17:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/hpilo.ko 2023-08-11T15:17:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29003.ko 2023-08-11T15:17:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29020.ko 2023-08-11T15:17:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d 2023-08-11T15:17:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-11T15:17:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-11T15:17:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei 2023-08-11T15:17:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-me.ko 2023-08-11T15:17:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-txe.ko 2023-08-11T15:17:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei.ko 2023-08-11T15:17:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru 2023-08-11T15:17:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-11T15:17:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp 2023-08-11T15:17:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-11T15:17:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-11T15:17:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-11T15:17:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_7xx1.ko 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_core.ko 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tsl2550.ko 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_balloon.ko 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding 2023-08-11T15:17:16.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding/bonding.ko 2023-08-11T15:17:16.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa 2023-08-11T15:17:16.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53 2023-08-11T15:17:16.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-11T15:17:16.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-11T15:17:16.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-11T15:17:16.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx 2023-08-11T15:17:16.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-11T15:17:16.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dummy.ko 2023-08-11T15:17:16.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/eql.ko 2023-08-11T15:17:16.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet 2023-08-11T15:17:16.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com 2023-08-11T15:17:16.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-11T15:17:16.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-11T15:17:16.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec 2023-08-11T15:17:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-11T15:17:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere 2023-08-11T15:17:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-11T15:17:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon 2023-08-11T15:17:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-11T15:17:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera 2023-08-11T15:17:16.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-11T15:17:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd 2023-08-11T15:17:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-11T15:17:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-11T15:17:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-11T15:17:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros 2023-08-11T15:17:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx 2023-08-11T15:17:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-11T15:17:16.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-11T15:17:16.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-11T15:17:16.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-11T15:17:16.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-11T15:17:16.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx 2023-08-11T15:17:16.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-11T15:17:16.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-11T15:17:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom 2023-08-11T15:17:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-11T15:17:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-11T15:17:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-11T15:17:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-11T15:17:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-11T15:17:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet 2023-08-11T15:17:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-11T15:17:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-11T15:17:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade 2023-08-11T15:17:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna 2023-08-11T15:17:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-11T15:17:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio 2023-08-11T15:17:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-11T15:17:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-11T15:17:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-11T15:17:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-11T15:17:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-11T15:17:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-11T15:17:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-11T15:17:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-11T15:17:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-11T15:17:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-11T15:17:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco 2023-08-11T15:17:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic 2023-08-11T15:17:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-11T15:17:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec 2023-08-11T15:17:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip 2023-08-11T15:17:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-11T15:17:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-11T15:17:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-11T15:17:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-11T15:17:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-11T15:17:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-11T15:17:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink 2023-08-11T15:17:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-11T15:17:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-11T15:17:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dnet.ko 2023-08-11T15:17:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex 2023-08-11T15:17:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet 2023-08-11T15:17:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-11T15:17:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ethoc.ko 2023-08-11T15:17:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/fealnx.ko 2023-08-11T15:17:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel 2023-08-11T15:17:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-11T15:17:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000 2023-08-11T15:17:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-11T15:17:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e 2023-08-11T15:17:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-11T15:17:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k 2023-08-11T15:17:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-11T15:17:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e 2023-08-11T15:17:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-11T15:17:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf 2023-08-11T15:17:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-11T15:17:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb 2023-08-11T15:17:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-11T15:17:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf 2023-08-11T15:17:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-11T15:17:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-11T15:17:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-11T15:17:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-11T15:17:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-11T15:17:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/jme.ko 2023-08-11T15:17:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell 2023-08-11T15:17:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-11T15:17:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-11T15:17:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-11T15:17:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox 2023-08-11T15:17:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-11T15:17:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-11T15:17:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-11T15:17:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-11T15:17:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-11T15:17:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-11T15:17:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel 2023-08-11T15:17:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-11T15:17:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom 2023-08-11T15:17:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-11T15:17:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-11T15:17:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi 2023-08-11T15:17:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-11T15:17:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-11T15:17:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion 2023-08-11T15:17:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-11T15:17:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia 2023-08-11T15:17:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-11T15:17:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines 2023-08-11T15:17:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-11T15:17:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-11T15:17:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic 2023-08-11T15:17:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-11T15:17:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-11T15:17:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-11T15:17:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-11T15:17:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-11T15:17:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc 2023-08-11T15:17:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-11T15:17:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek 2023-08-11T15:17:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-11T15:17:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-11T15:17:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-11T15:17:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-11T15:17:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker 2023-08-11T15:17:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-11T15:17:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc 2023-08-11T15:17:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-11T15:17:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan 2023-08-11T15:17:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-11T15:17:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis 2023-08-11T15:17:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-11T15:17:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-11T15:17:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc 2023-08-11T15:17:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-11T15:17:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-11T15:17:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-11T15:17:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-11T15:17:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro 2023-08-11T15:17:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-11T15:17:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-11T15:17:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-11T15:17:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun 2023-08-11T15:17:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-11T15:17:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-11T15:17:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-11T15:17:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-11T15:17:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti 2023-08-11T15:17:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-11T15:17:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti 2023-08-11T15:17:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-11T15:17:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via 2023-08-11T15:17:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-11T15:17:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-11T15:17:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet 2023-08-11T15:17:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-11T15:17:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-11T15:17:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom 2023-08-11T15:17:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-11T15:17:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/geneve.ko 2023-08-11T15:17:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv 2023-08-11T15:17:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-11T15:17:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154 2023-08-11T15:17:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-11T15:17:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ifb.ko 2023-08-11T15:17:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan 2023-08-11T15:17:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-11T15:17:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvlan.ko 2023-08-11T15:17:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvtap.ko 2023-08-11T15:17:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio 2023-08-11T15:17:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio.ko 2023-08-11T15:17:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-11T15:17:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-11T15:17:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-11T15:17:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mii.ko 2023-08-11T15:17:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/net_failover.ko 2023-08-11T15:17:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/netconsole.ko 2023-08-11T15:17:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/nlmon.ko 2023-08-11T15:17:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs 2023-08-11T15:17:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-11T15:17:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-11T15:17:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy 2023-08-11T15:17:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/amd.ko 2023-08-11T15:17:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-11T15:17:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-11T15:17:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-11T15:17:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm87xx.ko 2023-08-11T15:17:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/broadcom.ko 2023-08-11T15:17:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/cicada.ko 2023-08-11T15:17:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/davicom.ko 2023-08-11T15:17:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/dp83640.ko 2023-08-11T15:17:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/et1011c.ko 2023-08-11T15:17:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/icplus.ko 2023-08-11T15:17:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/lxt.ko 2023-08-11T15:17:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/marvell.ko 2023-08-11T15:17:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/micrel.ko 2023-08-11T15:17:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/national.ko 2023-08-11T15:17:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/phylink.ko 2023-08-11T15:17:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/qsemi.ko 2023-08-11T15:17:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/realtek.ko 2023-08-11T15:17:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/smsc.ko 2023-08-11T15:17:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/ste10Xp.ko 2023-08-11T15:17:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/vitesse.ko 2023-08-11T15:17:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp 2023-08-11T15:17:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-11T15:17:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_async.ko 2023-08-11T15:17:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-11T15:17:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-11T15:17:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-11T15:17:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-11T15:17:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppoe.ko 2023-08-11T15:17:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppox.ko 2023-08-11T15:17:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pptp.ko 2023-08-11T15:17:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip 2023-08-11T15:17:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slhc.ko 2023-08-11T15:17:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slip.ko 2023-08-11T15:17:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/sungem_phy.ko 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tap.ko 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team.ko 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_random.ko 2023-08-11T15:17:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-11T15:17:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tun.ko 2023-08-11T15:17:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/veth.ko 2023-08-11T15:17:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/virtio_net.ko 2023-08-11T15:17:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3 2023-08-11T15:17:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-11T15:17:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan 2023-08-11T15:17:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan/vxlan.ko 2023-08-11T15:17:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback 2023-08-11T15:17:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-11T15:17:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netfront.ko 2023-08-11T15:17:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb 2023-08-11T15:17:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb/ntb.ko 2023-08-11T15:17:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme 2023-08-11T15:17:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host 2023-08-11T15:17:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme-core.ko 2023-08-11T15:17:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme.ko 2023-08-11T15:17:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport 2023-08-11T15:17:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport.ko 2023-08-11T15:17:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_cs.ko 2023-08-11T15:17:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_pc.ko 2023-08-11T15:17:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_serial.ko 2023-08-11T15:17:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci 2023-08-11T15:17:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug 2023-08-11T15:17:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-11T15:17:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie 2023-08-11T15:17:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-11T15:17:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/xen-pcifront.ko 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/i82092.ko 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/pd6729.ko 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-11T15:17:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform 2023-08-11T15:17:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86 2023-08-11T15:17:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-11T15:17:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acerhdf.ko 2023-08-11T15:17:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-11T15:17:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-11T15:17:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/hdaps.ko 2023-08-11T15:17:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel 2023-08-11T15:17:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-11T15:17:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel_ips.ko 2023-08-11T15:17:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-11T15:17:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-11T15:17:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-11T15:17:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-11T15:17:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-11T15:17:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-11T15:17:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-11T15:17:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-11T15:17:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-11T15:17:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi.ko 2023-08-11T15:17:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap 2023-08-11T15:17:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-11T15:17:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-11T15:17:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps 2023-08-11T15:17:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients 2023-08-11T15:17:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-11T15:17:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-11T15:17:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps_parport.ko 2023-08-11T15:17:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/pps_core.ko 2023-08-11T15:17:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp 2023-08-11T15:17:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp.ko 2023-08-11T15:17:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp_kvm.ko 2023-08-11T15:17:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc 2023-08-11T15:17:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-11T15:17:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-11T15:17:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-11T15:17:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-11T15:17:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-11T15:17:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-11T15:17:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-11T15:17:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-11T15:17:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-11T15:17:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-11T15:17:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-11T15:17:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-em3027.ko 2023-08-11T15:17:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-11T15:17:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-11T15:17:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-11T15:17:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-11T15:17:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-11T15:17:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-11T15:17:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-max6900.ko 2023-08-11T15:17:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-11T15:17:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-11T15:17:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-11T15:17:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-11T15:17:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-11T15:17:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-11T15:17:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-11T15:17:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-11T15:17:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-11T15:17:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-11T15:17:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-11T15:17:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-11T15:17:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-x1205.ko 2023-08-11T15:17:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi 2023-08-11T15:17:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-9xxx.ko 2023-08-11T15:17:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-sas.ko 2023-08-11T15:17:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-xxxx.ko 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/BusLogic.ko 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/a100u2w.ko 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/advansys.ko 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-11T15:17:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-11T15:17:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/am53c974.ko 2023-08-11T15:17:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr 2023-08-11T15:17:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-11T15:17:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/atp870u.ko 2023-08-11T15:17:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi 2023-08-11T15:17:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-11T15:17:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa 2023-08-11T15:17:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa/bfa.ko 2023-08-11T15:17:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc 2023-08-11T15:17:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-11T15:17:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i 2023-08-11T15:17:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-11T15:17:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ch.ko 2023-08-11T15:17:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor 2023-08-11T15:17:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-11T15:17:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi 2023-08-11T15:17:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-11T15:17:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-11T15:17:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-11T15:17:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dc395x.ko 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-11T15:17:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dmx3191d.ko 2023-08-11T15:17:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r 2023-08-11T15:17:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-11T15:17:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esp_scsi.ko 2023-08-11T15:17:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe 2023-08-11T15:17:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-11T15:17:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-11T15:17:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic 2023-08-11T15:17:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic/fnic.ko 2023-08-11T15:17:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hpsa.ko 2023-08-11T15:17:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hptiop.ko 2023-08-11T15:17:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hv_storvsc.ko 2023-08-11T15:17:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/initio.ko 2023-08-11T15:17:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ipr.ko 2023-08-11T15:17:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ips.ko 2023-08-11T15:17:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci 2023-08-11T15:17:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci/isci.ko 2023-08-11T15:17:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-11T15:17:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-11T15:17:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc 2023-08-11T15:17:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc/libfc.ko 2023-08-11T15:17:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi.ko 2023-08-11T15:17:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-11T15:17:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas 2023-08-11T15:17:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas/libsas.ko 2023-08-11T15:17:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc 2023-08-11T15:17:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-11T15:17:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid 2023-08-11T15:17:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid.ko 2023-08-11T15:17:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-11T15:17:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-11T15:17:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-11T15:17:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas 2023-08-11T15:17:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-11T15:17:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas 2023-08-11T15:17:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-11T15:17:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvumi.ko 2023-08-11T15:17:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001 2023-08-11T15:17:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-11T15:17:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pmcraid.ko 2023-08-11T15:17:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla1280.ko 2023-08-11T15:17:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx 2023-08-11T15:17:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-11T15:17:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-11T15:17:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx 2023-08-11T15:17:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-11T15:17:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/raid_class.ko 2023-08-11T15:17:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_debug.ko 2023-08-11T15:17:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-11T15:17:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ses.ko 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/st.ko 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/stex.ko 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2 2023-08-11T15:17:17.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-11T15:17:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/virtio_scsi.ko 2023-08-11T15:17:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-11T15:17:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/wd719x.ko 2023-08-11T15:17:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb 2023-08-11T15:17:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb/ssb.ko 2023-08-11T15:17:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_file.ko 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_iblock.ko 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_mod.ko 2023-08-11T15:17:17.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_pscsi.ko 2023-08-11T15:17:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_user.ko 2023-08-11T15:17:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc 2023-08-11T15:17:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-11T15:17:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal 2023-08-11T15:17:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel 2023-08-11T15:17:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal 2023-08-11T15:17:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-11T15:17:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-11T15:17:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-11T15:17:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-11T15:17:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-11T15:17:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-11T15:17:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-11T15:17:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-11T15:17:17.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-11T15:17:17.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-11T15:17:17.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-11T15:17:17.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-11T15:17:17.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-11T15:17:17.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_gsm.ko 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_hdlc.ko 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/nozomi.ko 2023-08-11T15:17:17.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial 2023-08-11T15:17:17.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250 2023-08-11T15:17:17.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-11T15:17:17.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/arc_uart.ko 2023-08-11T15:17:17.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm 2023-08-11T15:17:17.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-11T15:17:17.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/synclink_gt.ko 2023-08-11T15:17:17.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs 2023-08-11T15:17:17.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core 2023-08-11T15:17:17.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-11T15:17:17.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host 2023-08-11T15:17:17.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-11T15:17:17.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio 2023-08-11T15:17:17.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio.ko 2023-08-11T15:17:17.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_aec.ko 2023-08-11T15:17:17.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_cif.ko 2023-08-11T15:17:17.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_pci_generic.ko 2023-08-11T15:17:17.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_sercos3.ko 2023-08-11T15:17:17.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb 2023-08-11T15:17:17.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class 2023-08-11T15:17:17.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-acm.ko 2023-08-11T15:17:17.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-11T15:17:17.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usblp.ko 2023-08-11T15:17:17.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usbtmc.ko 2023-08-11T15:17:17.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host 2023-08-11T15:17:17.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-11T15:17:17.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image 2023-08-11T15:17:17.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/mdc800.ko 2023-08-11T15:17:17.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/microtek.ko 2023-08-11T15:17:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc 2023-08-11T15:17:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/adutux.ko 2023-08-11T15:17:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/appledisplay.ko 2023-08-11T15:17:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi26.ko 2023-08-11T15:17:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi62.ko 2023-08-11T15:17:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ezusb.ko 2023-08-11T15:17:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/idmouse.ko 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/iowarrior.ko 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ldusb.ko 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/legousbtower.ko 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-11T15:17:17.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usb3503.ko 2023-08-11T15:17:17.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usblcd.ko 2023-08-11T15:17:17.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-11T15:17:17.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/uss720.ko 2023-08-11T15:17:17.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/yurex.ko 2023-08-11T15:17:17.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy 2023-08-11T15:17:17.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy/phy-generic.ko 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/aircable.ko 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ark3116.ko 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ch341.ko 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cp210x.ko 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cyberjack.ko 2023-08-11T15:17:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-11T15:17:17.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-11T15:17:17.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/empeg.ko 2023-08-11T15:17:17.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-11T15:17:17.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-11T15:17:17.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-11T15:17:17.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_ti.ko 2023-08-11T15:17:17.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipaq.ko 2023-08-11T15:17:17.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipw.ko 2023-08-11T15:17:17.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ir-usb.ko 2023-08-11T15:17:17.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-11T15:17:17.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan.ko 2023-08-11T15:17:17.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-11T15:17:17.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-11T15:17:17.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-11T15:17:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mct_u232.ko 2023-08-11T15:17:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7720.ko 2023-08-11T15:17:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7840.ko 2023-08-11T15:17:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/navman.ko 2023-08-11T15:17:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/omninet.ko 2023-08-11T15:17:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/opticon.ko 2023-08-11T15:17:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/option.ko 2023-08-11T15:17:17.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/oti6858.ko 2023-08-11T15:17:17.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/pl2303.ko 2023-08-11T15:17:17.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcaux.ko 2023-08-11T15:17:17.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcserial.ko 2023-08-11T15:17:17.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/quatech2.ko 2023-08-11T15:17:17.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/safe_serial.ko 2023-08-11T15:17:17.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/sierra.ko 2023-08-11T15:17:17.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-11T15:17:17.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ssu100.ko 2023-08-11T15:17:17.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/symbolserial.ko 2023-08-11T15:17:17.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-11T15:17:17.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-11T15:17:17.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_debug.ko 2023-08-11T15:17:17.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/visor.ko 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/whiteheat.ko 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/uas.ko 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-11T15:17:17.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-11T15:17:17.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-11T15:17:17.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-11T15:17:17.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-11T15:17:17.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-11T15:17:17.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-karma.ko 2023-08-11T15:17:17.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-11T15:17:17.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-11T15:17:17.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-11T15:17:17.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-11T15:17:17.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-11T15:17:17.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/usb-storage.ko 2023-08-11T15:17:17.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip 2023-08-11T15:17:17.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio.ko 2023-08-11T15:17:17.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-11T15:17:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost 2023-08-11T15:17:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost.ko 2023-08-11T15:17:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-11T15:17:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_net.ko 2023-08-11T15:17:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_scsi.ko 2023-08-11T15:17:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video 2023-08-11T15:17:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight 2023-08-11T15:17:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/apple_bl.ko 2023-08-11T15:17:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/lcd.ko 2023-08-11T15:17:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-11T15:17:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio 2023-08-11T15:17:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio.ko 2023-08-11T15:17:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_balloon.ko 2023-08-11T15:17:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_mmio.ko 2023-08-11T15:17:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci.ko 2023-08-11T15:17:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-11T15:17:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-11T15:17:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_ring.ko 2023-08-11T15:17:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1 2023-08-11T15:17:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters 2023-08-11T15:17:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2482.ko 2023-08-11T15:17:17.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2490.ko 2023-08-11T15:17:17.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves 2023-08-11T15:17:17.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-11T15:17:17.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-11T15:17:17.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-11T15:17:17.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-11T15:17:17.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-11T15:17:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-11T15:17:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-11T15:17:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-11T15:17:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-11T15:17:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-11T15:17:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-11T15:17:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/wire.ko 2023-08-11T15:17:17.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog 2023-08-11T15:17:17.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-11T15:17:17.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-11T15:17:17.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-11T15:17:17.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/hpwdt.ko 2023-08-11T15:17:17.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/i6300esb.ko 2023-08-11T15:17:17.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-11T15:17:17.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-11T15:17:17.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ib700wdt.ko 2023-08-11T15:17:17.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ibmasr.ko 2023-08-11T15:17:17.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-11T15:17:17.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-11T15:17:17.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it87_wdt.ko 2023-08-11T15:17:17.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/machzwd.ko 2023-08-11T15:17:17.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/nv_tco.ko 2023-08-11T15:17:17.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-11T15:17:17.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-11T15:17:17.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-11T15:17:17.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-11T15:17:17.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/softdog.ko 2023-08-11T15:17:17.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/via_wdt.ko 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/wdt_pci.ko 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/xen_wdt.ko 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen 2023-08-11T15:17:17.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-11T15:17:17.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-evtchn.ko 2023-08-11T15:17:17.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntalloc.ko 2023-08-11T15:17:17.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntdev.ko 2023-08-11T15:17:17.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback 2023-08-11T15:17:17.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-11T15:17:17.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-privcmd.ko 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p/9p.ko 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs/affs.ko 2023-08-11T15:17:17.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs 2023-08-11T15:17:17.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs/befs.ko 2023-08-11T15:17:17.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/binfmt_misc.ko 2023-08-11T15:17:17.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs 2023-08-11T15:17:17.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs/btrfs.ko 2023-08-11T15:17:17.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles 2023-08-11T15:17:17.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles/cachefiles.ko 2023-08-11T15:17:17.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph 2023-08-11T15:17:17.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph/ceph.ko 2023-08-11T15:17:17.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda 2023-08-11T15:17:17.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda/coda.ko 2023-08-11T15:17:17.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs 2023-08-11T15:17:17.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs/cramfs.ko 2023-08-11T15:17:17.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm 2023-08-11T15:17:17.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm/dlm.ko 2023-08-11T15:17:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs 2023-08-11T15:17:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-11T15:17:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs 2023-08-11T15:17:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs/f2fs.ko 2023-08-11T15:17:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat 2023-08-11T15:17:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/fat.ko 2023-08-11T15:17:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/msdos.ko 2023-08-11T15:17:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/vfat.ko 2023-08-11T15:17:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache 2023-08-11T15:17:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache/fscache.ko 2023-08-11T15:17:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse 2023-08-11T15:17:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/cuse.ko 2023-08-11T15:17:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/fuse.ko 2023-08-11T15:17:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2 2023-08-11T15:17:17.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2/gfs2.ko 2023-08-11T15:17:17.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs 2023-08-11T15:17:17.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs/hfs.ko 2023-08-11T15:17:17.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus 2023-08-11T15:17:17.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus/hfsplus.ko 2023-08-11T15:17:17.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs 2023-08-11T15:17:17.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs/isofs.ko 2023-08-11T15:17:17.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs 2023-08-11T15:17:17.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs/jfs.ko 2023-08-11T15:17:17.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd 2023-08-11T15:17:17.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd/lockd.ko 2023-08-11T15:17:17.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix 2023-08-11T15:17:17.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix/minix.ko 2023-08-11T15:17:17.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs 2023-08-11T15:17:17.096 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs/netfs.ko 2023-08-11T15:17:17.096 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs 2023-08-11T15:17:17.096 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout 2023-08-11T15:17:17.096 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-11T15:17:17.096 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout 2023-08-11T15:17:17.096 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-11T15:17:17.096 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout 2023-08-11T15:17:17.097 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-11T15:17:17.097 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfs.ko 2023-08-11T15:17:17.097 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv3.ko 2023-08-11T15:17:17.097 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv4.ko 2023-08-11T15:17:17.097 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common 2023-08-11T15:17:17.097 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/grace.ko 2023-08-11T15:17:17.097 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/nfs_acl.ko 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd/nfsd.ko 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2/nilfs2.ko 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-celtic.ko 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-centeuro.ko 2023-08-11T15:17:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-croatian.ko 2023-08-11T15:17:17.099 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-cyrillic.ko 2023-08-11T15:17:17.099 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-gaelic.ko 2023-08-11T15:17:17.099 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-greek.ko 2023-08-11T15:17:17.099 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-iceland.ko 2023-08-11T15:17:17.099 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-inuit.ko 2023-08-11T15:17:17.099 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-roman.ko 2023-08-11T15:17:17.099 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-romanian.ko 2023-08-11T15:17:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-turkish.ko 2023-08-11T15:17:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1250.ko 2023-08-11T15:17:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1251.ko 2023-08-11T15:17:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1255.ko 2023-08-11T15:17:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp737.ko 2023-08-11T15:17:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp775.ko 2023-08-11T15:17:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp850.ko 2023-08-11T15:17:17.101 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp852.ko 2023-08-11T15:17:17.101 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp855.ko 2023-08-11T15:17:17.101 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp857.ko 2023-08-11T15:17:17.101 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp860.ko 2023-08-11T15:17:17.101 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp861.ko 2023-08-11T15:17:17.101 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp862.ko 2023-08-11T15:17:17.101 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp863.ko 2023-08-11T15:17:17.102 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp864.ko 2023-08-11T15:17:17.102 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp865.ko 2023-08-11T15:17:17.102 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp866.ko 2023-08-11T15:17:17.102 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp869.ko 2023-08-11T15:17:17.102 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp874.ko 2023-08-11T15:17:17.102 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp932.ko 2023-08-11T15:17:17.102 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp936.ko 2023-08-11T15:17:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp949.ko 2023-08-11T15:17:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp950.ko 2023-08-11T15:17:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_euc-jp.ko 2023-08-11T15:17:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-1.ko 2023-08-11T15:17:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-13.ko 2023-08-11T15:17:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-14.ko 2023-08-11T15:17:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-15.ko 2023-08-11T15:17:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-2.ko 2023-08-11T15:17:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-3.ko 2023-08-11T15:17:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-4.ko 2023-08-11T15:17:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-5.ko 2023-08-11T15:17:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-6.ko 2023-08-11T15:17:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-7.ko 2023-08-11T15:17:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-9.ko 2023-08-11T15:17:17.105 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-r.ko 2023-08-11T15:17:17.105 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-ru.ko 2023-08-11T15:17:17.105 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-u.ko 2023-08-11T15:17:17.105 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_utf8.ko 2023-08-11T15:17:17.105 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2 2023-08-11T15:17:17.105 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster 2023-08-11T15:17:17.105 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2.ko 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-11T15:17:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-11T15:17:17.107 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs 2023-08-11T15:17:17.107 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs/overlay.ko 2023-08-11T15:17:17.107 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore 2023-08-11T15:17:17.107 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore/ramoops.ko 2023-08-11T15:17:17.107 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs 2023-08-11T15:17:17.107 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs/reiserfs.ko 2023-08-11T15:17:17.107 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs 2023-08-11T15:17:17.108 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs/romfs.ko 2023-08-11T15:17:17.108 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb 2023-08-11T15:17:17.108 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client 2023-08-11T15:17:17.108 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client/cifs.ko 2023-08-11T15:17:17.108 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common 2023-08-11T15:17:17.108 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_arc4.ko 2023-08-11T15:17:17.108 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_md4.ko 2023-08-11T15:17:17.109 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs 2023-08-11T15:17:17.109 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs/squashfs.ko 2023-08-11T15:17:17.109 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv 2023-08-11T15:17:17.109 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv/sysv.ko 2023-08-11T15:17:17.109 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf 2023-08-11T15:17:17.109 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf/udf.ko 2023-08-11T15:17:17.109 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs 2023-08-11T15:17:17.110 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs/ufs.ko 2023-08-11T15:17:17.110 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs 2023-08-11T15:17:17.110 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs/xfs.ko 2023-08-11T15:17:17.110 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel 2023-08-11T15:17:17.110 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu 2023-08-11T15:17:17.110 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu/rcutorture.ko 2023-08-11T15:17:17.110 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/torture.ko 2023-08-11T15:17:17.111 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace 2023-08-11T15:17:17.111 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-11T15:17:17.111 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib 2023-08-11T15:17:17.111 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/asn1_encoder.ko 2023-08-11T15:17:17.111 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc-itu-t.ko 2023-08-11T15:17:17.111 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc8.ko 2023-08-11T15:17:17.111 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto 2023-08-11T15:17:17.112 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libarc4.ko 2023-08-11T15:17:17.112 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libdes.ko 2023-08-11T15:17:17.112 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/libcrc32c.ko 2023-08-11T15:17:17.112 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lru_cache.ko 2023-08-11T15:17:17.112 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4 2023-08-11T15:17:17.112 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4_compress.ko 2023-08-11T15:17:17.112 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4hc_compress.ko 2023-08-11T15:17:17.113 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math 2023-08-11T15:17:17.113 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math/cordic.ko 2023-08-11T15:17:17.113 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6 2023-08-11T15:17:17.113 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6/raid6_pq.ko 2023-08-11T15:17:17.113 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon 2023-08-11T15:17:17.113 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-11T15:17:17.113 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_bm.ko 2023-08-11T15:17:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_fsm.ko 2023-08-11T15:17:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_kmp.ko 2023-08-11T15:17:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd 2023-08-11T15:17:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd/zstd_compress.ko 2023-08-11T15:17:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm 2023-08-11T15:17:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm/hwpoison-inject.ko 2023-08-11T15:17:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net 2023-08-11T15:17:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan 2023-08-11T15:17:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/6lowpan.ko 2023-08-11T15:17:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_dest.ko 2023-08-11T15:17:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_fragment.ko 2023-08-11T15:17:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_hop.ko 2023-08-11T15:17:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-11T15:17:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_mobility.ko 2023-08-11T15:17:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_routing.ko 2023-08-11T15:17:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_udp.ko 2023-08-11T15:17:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802 2023-08-11T15:17:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/garp.ko 2023-08-11T15:17:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/mrp.ko 2023-08-11T15:17:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/p8022.ko 2023-08-11T15:17:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/psnap.ko 2023-08-11T15:17:17.117 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/stp.ko 2023-08-11T15:17:17.117 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q 2023-08-11T15:17:17.117 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q/8021q.ko 2023-08-11T15:17:17.117 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p 2023-08-11T15:17:17.117 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet.ko 2023-08-11T15:17:17.117 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_fd.ko 2023-08-11T15:17:17.117 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_rdma.ko 2023-08-11T15:17:17.118 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_virtio.ko 2023-08-11T15:17:17.118 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv 2023-08-11T15:17:17.118 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv/batman-adv.ko 2023-08-11T15:17:17.118 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge 2023-08-11T15:17:17.118 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/br_netfilter.ko 2023-08-11T15:17:17.118 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/bridge.ko 2023-08-11T15:17:17.118 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter 2023-08-11T15:17:17.119 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-11T15:17:17.119 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-11T15:17:17.119 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-11T15:17:17.119 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-11T15:17:17.119 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-11T15:17:17.119 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-11T15:17:17.119 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-11T15:17:17.120 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-11T15:17:17.120 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-11T15:17:17.120 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-11T15:17:17.120 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-11T15:17:17.120 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-11T15:17:17.120 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-11T15:17:17.120 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtables.ko 2023-08-11T15:17:17.121 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-11T15:17:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-11T15:17:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-11T15:17:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph 2023-08-11T15:17:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph/libceph.ko 2023-08-11T15:17:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core 2023-08-11T15:17:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/failover.ko 2023-08-11T15:17:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/pktgen.ko 2023-08-11T15:17:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp 2023-08-11T15:17:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp.ko 2023-08-11T15:17:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_diag.ko 2023-08-11T15:17:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv4.ko 2023-08-11T15:17:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv6.ko 2023-08-11T15:17:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver 2023-08-11T15:17:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver/dns_resolver.ko 2023-08-11T15:17:17.124 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa 2023-08-11T15:17:17.124 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/dsa_core.ko 2023-08-11T15:17:17.124 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_brcm.ko 2023-08-11T15:17:17.124 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_dsa.ko 2023-08-11T15:17:17.124 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_none.ko 2023-08-11T15:17:17.124 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_trailer.ko 2023-08-11T15:17:17.124 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154 2023-08-11T15:17:17.125 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan 2023-08-11T15:17:17.125 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-11T15:17:17.125 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154.ko 2023-08-11T15:17:17.125 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-11T15:17:17.125 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4 2023-08-11T15:17:17.125 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ah4.ko 2023-08-11T15:17:17.125 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/esp4.ko 2023-08-11T15:17:17.126 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/fou.ko 2023-08-11T15:17:17.126 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/gre.ko 2023-08-11T15:17:17.126 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/inet_diag.ko 2023-08-11T15:17:17.126 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_gre.ko 2023-08-11T15:17:17.126 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_tunnel.ko 2023-08-11T15:17:17.126 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_vti.ko 2023-08-11T15:17:17.126 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipcomp.ko 2023-08-11T15:17:17.127 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipip.ko 2023-08-11T15:17:17.127 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter 2023-08-11T15:17:17.127 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-11T15:17:17.127 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-11T15:17:17.127 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-11T15:17:17.127 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-11T15:17:17.127 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-11T15:17:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-11T15:17:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-11T15:17:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-11T15:17:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-11T15:17:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-11T15:17:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-11T15:17:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-11T15:17:17.129 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-11T15:17:17.129 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-11T15:17:17.129 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-11T15:17:17.129 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-11T15:17:17.129 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-11T15:17:17.129 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-11T15:17:17.129 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-11T15:17:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-11T15:17:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-11T15:17:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_bic.ko 2023-08-11T15:17:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_dctcp.ko 2023-08-11T15:17:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_diag.ko 2023-08-11T15:17:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_highspeed.ko 2023-08-11T15:17:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_htcp.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_hybla.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_illinois.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_lp.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_scalable.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_vegas.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_veno.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_westwood.ko 2023-08-11T15:17:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_yeah.ko 2023-08-11T15:17:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tunnel4.ko 2023-08-11T15:17:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_diag.ko 2023-08-11T15:17:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_tunnel.ko 2023-08-11T15:17:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-11T15:17:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6 2023-08-11T15:17:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ah6.ko 2023-08-11T15:17:17.132 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/esp6.ko 2023-08-11T15:17:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/fou6.ko 2023-08-11T15:17:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_tunnel.ko 2023-08-11T15:17:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-11T15:17:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_vti.ko 2023-08-11T15:17:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ipcomp6.ko 2023-08-11T15:17:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter 2023-08-11T15:17:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-11T15:17:17.134 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-11T15:17:17.134 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-11T15:17:17.134 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-11T15:17:17.134 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-11T15:17:17.134 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-11T15:17:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-11T15:17:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-11T15:17:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-11T15:17:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-11T15:17:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-11T15:17:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-11T15:17:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-11T15:17:17.136 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-11T15:17:17.136 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-11T15:17:17.136 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-11T15:17:17.136 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-11T15:17:17.136 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-11T15:17:17.136 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-11T15:17:17.136 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-11T15:17:17.137 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-11T15:17:17.137 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-11T15:17:17.137 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-11T15:17:17.137 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-11T15:17:17.137 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/sit.ko 2023-08-11T15:17:17.137 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/tunnel6.ko 2023-08-11T15:17:17.137 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-11T15:17:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key 2023-08-11T15:17:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key/af_key.ko 2023-08-11T15:17:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp 2023-08-11T15:17:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_core.ko 2023-08-11T15:17:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-11T15:17:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_eth.ko 2023-08-11T15:17:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip.ko 2023-08-11T15:17:17.139 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip6.ko 2023-08-11T15:17:17.139 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_netlink.ko 2023-08-11T15:17:17.139 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ppp.ko 2023-08-11T15:17:17.139 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc 2023-08-11T15:17:17.139 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc/llc.ko 2023-08-11T15:17:17.139 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154 2023-08-11T15:17:17.139 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154/mac802154.ko 2023-08-11T15:17:17.140 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls 2023-08-11T15:17:17.140 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls/mpls_gso.ko 2023-08-11T15:17:17.140 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter 2023-08-11T15:17:17.140 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset 2023-08-11T15:17:17.140 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set.ko 2023-08-11T15:17:17.140 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-11T15:17:17.140 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-11T15:17:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-11T15:17:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-11T15:17:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-11T15:17:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-11T15:17:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-11T15:17:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-11T15:17:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-11T15:17:17.142 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-11T15:17:17.142 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-11T15:17:17.142 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-11T15:17:17.142 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-11T15:17:17.142 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-11T15:17:17.142 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-11T15:17:17.142 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs 2023-08-11T15:17:17.143 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-11T15:17:17.143 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-11T15:17:17.143 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-11T15:17:17.143 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-11T15:17:17.143 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-11T15:17:17.143 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-11T15:17:17.143 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-11T15:17:17.144 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-11T15:17:17.144 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-11T15:17:17.144 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-11T15:17:17.144 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-11T15:17:17.144 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-11T15:17:17.144 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-11T15:17:17.144 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-11T15:17:17.145 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conncount.ko 2023-08-11T15:17:17.145 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack.ko 2023-08-11T15:17:17.145 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-11T15:17:17.145 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-11T15:17:17.145 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-11T15:17:17.145 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-11T15:17:17.145 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-11T15:17:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-11T15:17:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-11T15:17:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-11T15:17:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-11T15:17:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-11T15:17:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-11T15:17:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-11T15:17:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-11T15:17:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table.ko 2023-08-11T15:17:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-11T15:17:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_log_syslog.ko 2023-08-11T15:17:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat.ko 2023-08-11T15:17:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-11T15:17:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-11T15:17:17.148 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_irc.ko 2023-08-11T15:17:17.148 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_sip.ko 2023-08-11T15:17:17.148 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-11T15:17:17.148 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-11T15:17:17.148 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_tables.ko 2023-08-11T15:17:17.148 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink.ko 2023-08-11T15:17:17.148 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-11T15:17:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_log.ko 2023-08-11T15:17:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-11T15:17:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-11T15:17:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_chain_nat.ko 2023-08-11T15:17:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_compat.ko 2023-08-11T15:17:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_ct.ko 2023-08-11T15:17:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fib.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_hash.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_limit.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_log.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_masq.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_nat.ko 2023-08-11T15:17:17.150 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_numgen.ko 2023-08-11T15:17:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_queue.ko 2023-08-11T15:17:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_quota.ko 2023-08-11T15:17:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_redir.ko 2023-08-11T15:17:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject.ko 2023-08-11T15:17:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject_inet.ko 2023-08-11T15:17:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_socket.ko 2023-08-11T15:17:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_tunnel.ko 2023-08-11T15:17:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_AUDIT.ko 2023-08-11T15:17:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-11T15:17:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-11T15:17:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-11T15:17:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CT.ko 2023-08-11T15:17:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_DSCP.ko 2023-08-11T15:17:17.152 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HL.ko 2023-08-11T15:17:17.153 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HMARK.ko 2023-08-11T15:17:17.153 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-11T15:17:17.153 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LED.ko 2023-08-11T15:17:17.153 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LOG.ko 2023-08-11T15:17:17.153 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-11T15:17:17.153 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NETMAP.ko 2023-08-11T15:17:17.153 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFLOG.ko 2023-08-11T15:17:17.154 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-11T15:17:17.154 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_RATEEST.ko 2023-08-11T15:17:17.154 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-11T15:17:17.154 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_SECMARK.ko 2023-08-11T15:17:17.154 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-11T15:17:17.154 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-11T15:17:17.154 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TEE.ko 2023-08-11T15:17:17.155 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TPROXY.ko 2023-08-11T15:17:17.155 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TRACE.ko 2023-08-11T15:17:17.155 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_addrtype.ko 2023-08-11T15:17:17.155 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_bpf.ko 2023-08-11T15:17:17.155 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cgroup.ko 2023-08-11T15:17:17.155 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cluster.ko 2023-08-11T15:17:17.155 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_comment.ko 2023-08-11T15:17:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connbytes.ko 2023-08-11T15:17:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlabel.ko 2023-08-11T15:17:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlimit.ko 2023-08-11T15:17:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connmark.ko 2023-08-11T15:17:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_conntrack.ko 2023-08-11T15:17:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cpu.ko 2023-08-11T15:17:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dccp.ko 2023-08-11T15:17:17.157 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_devgroup.ko 2023-08-11T15:17:17.157 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dscp.ko 2023-08-11T15:17:17.157 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ecn.ko 2023-08-11T15:17:17.157 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_esp.ko 2023-08-11T15:17:17.157 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hashlimit.ko 2023-08-11T15:17:17.157 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_helper.ko 2023-08-11T15:17:17.157 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hl.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipcomp.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_iprange.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipvs.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_l2tp.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_length.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_limit.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mac.ko 2023-08-11T15:17:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mark.ko 2023-08-11T15:17:17.159 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_multiport.ko 2023-08-11T15:17:17.159 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nat.ko 2023-08-11T15:17:17.159 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nfacct.ko 2023-08-11T15:17:17.159 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_osf.ko 2023-08-11T15:17:17.159 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_owner.ko 2023-08-11T15:17:17.159 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_physdev.ko 2023-08-11T15:17:17.159 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_pkttype.ko 2023-08-11T15:17:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_policy.ko 2023-08-11T15:17:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_quota.ko 2023-08-11T15:17:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_rateest.ko 2023-08-11T15:17:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_realm.ko 2023-08-11T15:17:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_recent.ko 2023-08-11T15:17:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_sctp.ko 2023-08-11T15:17:17.160 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_set.ko 2023-08-11T15:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_socket.ko 2023-08-11T15:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_state.ko 2023-08-11T15:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_statistic.ko 2023-08-11T15:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_string.ko 2023-08-11T15:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_tcpmss.ko 2023-08-11T15:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_time.ko 2023-08-11T15:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_u32.ko 2023-08-11T15:17:17.162 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink 2023-08-11T15:17:17.162 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink/netlink_diag.ko 2023-08-11T15:17:17.162 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh 2023-08-11T15:17:17.162 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh/nsh.ko 2023-08-11T15:17:17.162 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch 2023-08-11T15:17:17.162 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/openvswitch.ko 2023-08-11T15:17:17.162 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-geneve.ko 2023-08-11T15:17:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-gre.ko 2023-08-11T15:17:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-vxlan.ko 2023-08-11T15:17:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet 2023-08-11T15:17:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet/af_packet_diag.ko 2023-08-11T15:17:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds 2023-08-11T15:17:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds.ko 2023-08-11T15:17:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_rdma.ko 2023-08-11T15:17:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_tcp.ko 2023-08-11T15:17:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched 2023-08-11T15:17:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_csum.ko 2023-08-11T15:17:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_gact.ko 2023-08-11T15:17:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_ipt.ko 2023-08-11T15:17:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_mirred.ko 2023-08-11T15:17:17.164 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_nat.ko 2023-08-11T15:17:17.165 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_pedit.ko 2023-08-11T15:17:17.165 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_police.ko 2023-08-11T15:17:17.165 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_simple.ko 2023-08-11T15:17:17.165 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_skbedit.ko 2023-08-11T15:17:17.165 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_vlan.ko 2023-08-11T15:17:17.165 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_basic.ko 2023-08-11T15:17:17.165 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_bpf.ko 2023-08-11T15:17:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_flow.ko 2023-08-11T15:17:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_fw.ko 2023-08-11T15:17:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_route.ko 2023-08-11T15:17:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_u32.ko 2023-08-11T15:17:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_cmp.ko 2023-08-11T15:17:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_ipset.ko 2023-08-11T15:17:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_meta.ko 2023-08-11T15:17:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_nbyte.ko 2023-08-11T15:17:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_text.ko 2023-08-11T15:17:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_u32.ko 2023-08-11T15:17:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_choke.ko 2023-08-11T15:17:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_codel.ko 2023-08-11T15:17:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_drr.ko 2023-08-11T15:17:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_fq.ko 2023-08-11T15:17:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_gred.ko 2023-08-11T15:17:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hfsc.ko 2023-08-11T15:17:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hhf.ko 2023-08-11T15:17:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_htb.ko 2023-08-11T15:17:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_ingress.ko 2023-08-11T15:17:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio.ko 2023-08-11T15:17:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio_lib.ko 2023-08-11T15:17:17.169 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_multiq.ko 2023-08-11T15:17:17.169 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_netem.ko 2023-08-11T15:17:17.169 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_pie.ko 2023-08-11T15:17:17.169 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_plug.ko 2023-08-11T15:17:17.169 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_prio.ko 2023-08-11T15:17:17.169 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_qfq.ko 2023-08-11T15:17:17.169 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_red.ko 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfb.ko 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfq.ko 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_tbf.ko 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_teql.ko 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp.ko 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp_diag.ko 2023-08-11T15:17:17.170 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc 2023-08-11T15:17:17.171 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss 2023-08-11T15:17:17.171 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-11T15:17:17.171 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-11T15:17:17.171 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/sunrpc.ko 2023-08-11T15:17:17.171 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma 2023-08-11T15:17:17.171 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-11T15:17:17.171 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc 2023-08-11T15:17:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/diag.ko 2023-08-11T15:17:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/tipc.ko 2023-08-11T15:17:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix 2023-08-11T15:17:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix/unix_diag.ko 2023-08-11T15:17:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock 2023-08-11T15:17:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-11T15:17:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-11T15:17:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock.ko 2023-08-11T15:17:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-11T15:17:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-11T15:17:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm 2023-08-11T15:17:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-11T15:17:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security 2023-08-11T15:17:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys 2023-08-11T15:17:17.174 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys 2023-08-11T15:17:17.174 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-11T15:17:17.174 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys 2023-08-11T15:17:17.174 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys/trusted.ko 2023-08-11T15:17:17.174 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt 2023-08-11T15:17:17.174 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib 2023-08-11T15:17:17.174 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib/irqbypass.ko 2023-08-11T15:17:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias 2023-08-11T15:17:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias.bin 2023-08-11T15:17:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin 2023-08-11T15:17:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.bin 2023-08-11T15:17:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.modinfo 2023-08-11T15:17:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep 2023-08-11T15:17:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep.bin 2023-08-11T15:17:17.176 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.devname 2023-08-11T15:17:17.176 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.order 2023-08-11T15:17:17.176 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.softdep 2023-08-11T15:17:17.176 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols 2023-08-11T15:17:17.176 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols.bin 2023-08-11T15:17:17.177 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:17.177 DEBUG:teuthology.orchestra.run.smithi032:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-11T15:17:17.205 INFO:teuthology.orchestra.run.smithi032.stdout:postinstall scriptlet (using /bin/sh): 2023-08-11T15:17:17.206 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d -a -r /boot/System.map-6.5.0-rc4-ga7fb1265323d ]; then 2023-08-11T15:17:17.206 INFO:teuthology.orchestra.run.smithi032.stdout:cp /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:17:17.206 INFO:teuthology.orchestra.run.smithi032.stdout:cp /boot/System.map-6.5.0-rc4-ga7fb1265323d /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:17:17.206 INFO:teuthology.orchestra.run.smithi032.stdout:rm -f /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:17.207 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/installkernel 6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:17:17.207 INFO:teuthology.orchestra.run.smithi032.stdout:rm -f /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T15:17:17.207 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2023-08-11T15:17:17.207 INFO:teuthology.orchestra.run.smithi032.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-11T15:17:17.207 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-11T15:17:17.207 INFO:teuthology.orchestra.run.smithi032.stdout:new-kernel-pkg --remove 6.5.0-rc4-ga7fb1265323d --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc4-ga7fb1265323d.img 2023-08-11T15:17:17.208 INFO:teuthology.orchestra.run.smithi032.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-11T15:17:17.208 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-install remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:17.208 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2023-08-11T15:17:17.208 INFO:teuthology.orchestra.run.smithi032.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-11T15:17:17.208 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-11T15:17:17.209 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/update-bootloader --remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:17.209 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2023-08-11T15:17:17.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -qi grub2-tools 2023-08-11T15:17:17.290 INFO:teuthology.orchestra.run.smithi032.stdout:Name : grub2-tools 2023-08-11T15:17:17.290 INFO:teuthology.orchestra.run.smithi032.stdout:Epoch : 1 2023-08-11T15:17:17.290 INFO:teuthology.orchestra.run.smithi032.stdout:Version : 2.02 2023-08-11T15:17:17.291 INFO:teuthology.orchestra.run.smithi032.stdout:Release : 123.el8 2023-08-11T15:17:17.292 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: x86_64 2023-08-11T15:17:17.292 INFO:teuthology.orchestra.run.smithi032.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-08-11T15:17:17.293 INFO:teuthology.orchestra.run.smithi032.stdout:Group : System Environment/Base 2023-08-11T15:17:17.293 INFO:teuthology.orchestra.run.smithi032.stdout:Size : 9265450 2023-08-11T15:17:17.293 INFO:teuthology.orchestra.run.smithi032.stdout:License : GPLv3+ 2023-08-11T15:17:17.294 INFO:teuthology.orchestra.run.smithi032.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-08-11T15:17:17.294 INFO:teuthology.orchestra.run.smithi032.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-08-11T15:17:17.294 INFO:teuthology.orchestra.run.smithi032.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-08-11T15:17:17.294 INFO:teuthology.orchestra.run.smithi032.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-08-11T15:17:17.295 INFO:teuthology.orchestra.run.smithi032.stdout:Relocations : (not relocatable) 2023-08-11T15:17:17.295 INFO:teuthology.orchestra.run.smithi032.stdout:Packager : Red Hat, Inc. 2023-08-11T15:17:17.295 INFO:teuthology.orchestra.run.smithi032.stdout:Vendor : Red Hat, Inc. 2023-08-11T15:17:17.295 INFO:teuthology.orchestra.run.smithi032.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-11T15:17:17.296 INFO:teuthology.orchestra.run.smithi032.stdout:Summary : Support tools for GRUB. 2023-08-11T15:17:17.296 INFO:teuthology.orchestra.run.smithi032.stdout:Description : 2023-08-11T15:17:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:17:17.296 INFO:teuthology.orchestra.run.smithi032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-11T15:17:17.297 INFO:teuthology.orchestra.run.smithi032.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-11T15:17:17.297 INFO:teuthology.orchestra.run.smithi032.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-11T15:17:17.297 INFO:teuthology.orchestra.run.smithi032.stdout:hardware devices. 2023-08-11T15:17:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:17:17.297 INFO:teuthology.orchestra.run.smithi032.stdout:This subpackage provides tools for support of all platforms. 2023-08-11T15:17:17.298 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-11T15:17:17.299 INFO:teuthology.task.kernel:Updating grub on smithi032 to boot 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:17.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-11T15:17:18.623 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-08-11T15:17:20.847 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-08-11T15:17:20.849 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-08-11T15:17:20.876 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.MSHb6uwa5e 2023-08-11T15:17:20.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MSHb6uwa5e 2023-08-11T15:17:20.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.MSHb6uwa5e 2023-08-11T15:17:21.087 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.MSHb6uwa5e is 6KB 2023-08-11T15:17:21.136 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.MSHb6uwa5e 2023-08-11T15:17:21.153 DEBUG:teuthology.orchestra.run.smithi032:> sudo /bin/ls /boot/loader/entries || true 2023-08-11T15:17:21.231 INFO:teuthology.orchestra.run.smithi032.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-08-11T15:17:21.232 INFO:teuthology.orchestra.run.smithi032.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-08-11T15:17:21.232 INFO:teuthology.orchestra.run.smithi032.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc4-ga7fb1265323d.conf 2023-08-11T15:17:21.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc4-ga7fb1265323d 2023-08-11T15:17:21.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2023-08-11T15:17:51.397 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-08-11T15:17:51.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-08-11T15:17:51.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:18:09.776 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-08-11T15:18:18.778 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-08-11T15:18:18.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:18:37.172 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-08-11T15:18:49.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-08-11T15:18:49.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:18:52.240 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-08-11T15:19:07.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-08-11T15:19:07.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:19:07.597 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-08-11T15:19:07.976 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-08-11T15:19:07.976 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc4-ga7fb1265323d"... 2023-08-11T15:19:07.976 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-08-11T15:19:08.019 INFO:teuthology.orchestra.run.smithi032.stdout:6.5.0-rc4-ga7fb1265323d 2023-08-11T15:19:08.019 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc4-ga7fb1265323d vs 6.5.0-rc4-ga7fb1265323d 2023-08-11T15:19:08.019 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-11T15:19:08.019 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-08-11T15:19:09.020 DEBUG:teuthology.parallel:result is None 2023-08-11T15:19:09.021 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-11T15:19:09.030 INFO:teuthology.task.internal:Creating test directory... 2023-08-11T15:19:09.031 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T15:19:09.036 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T15:19:09.038 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T15:19:09.078 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-11T15:19:09.085 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-11T15:19:09.091 INFO:teuthology.task.internal:Creating archive directory... 2023-08-11T15:19:09.092 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T15:19:09.097 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T15:19:09.099 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T15:19:09.199 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-11T15:19:09.206 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-11T15:19:09.206 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T15:19:09.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:19:09.225 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T15:19:09.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:19:09.244 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T15:19:09.249 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T15:19:09.268 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T15:19:09.312 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:19:09.334 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:19:09.335 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:19:09.351 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T15:19:09.420 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-11T15:19:09.430 INFO:teuthology.task.internal:Configuring sudo... 2023-08-11T15:19:09.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T15:19:09.435 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T15:19:09.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T15:19:09.501 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-11T15:19:09.512 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-11T15:19:09.513 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T15:19:09.517 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T15:19:09.520 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T15:19:09.555 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:19:09.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:19:09.753 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:19:09.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:19:09.907 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:19:09.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T15:19:09.978 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:19:10.024 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:19:10.084 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:19:10.170 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:19:10.218 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:19:10.219 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T15:19:10.291 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:19:10.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T15:19:10.400 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:19:10.444 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T15:19:10.489 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:19:10.490 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T15:19:10.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-08-11T15:19:10.563 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2023-08-11T15:19:10.566 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2023-08-11T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T15:19:10.665 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T15:19:10.676 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T15:19:11.089 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-11T15:19:11.096 INFO:teuthology.task.internal:Starting timer... 2023-08-11T15:19:11.097 INFO:teuthology.run_tasks:Running task pcp... 2023-08-11T15:19:11.108 INFO:teuthology.run_tasks:Running task selinux... 2023-08-11T15:19:11.119 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-08-11T15:19:11.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2023-08-11T15:19:11.253 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2023-08-11T15:19:11.273 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2023-08-11T15:19:11.344 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2023-08-11T15:19:11.346 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2023-08-11T15:19:11.412 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2023-08-11T15:19:11.414 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-11T15:19:11.414 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2023-08-11T15:19:11.461 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2023-08-11T15:19:11.462 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2023-08-11T15:19:11.486 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2023-08-11T15:19:11.487 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2023-08-11T15:19:11.512 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2023-08-11T15:19:11.513 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2023-08-11T15:19:11.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T15:19:11.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:19:11.586 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T15:19:11.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:19:11.615 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T15:19:11.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:19:11.643 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-11T15:19:11.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2023-08-11T15:19:11.714 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2023-08-11T15:19:11.741 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2023-08-11T15:19:11.768 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-11T15:19:11.777 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-08-11T15:19:11.778 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-11T15:19:11.801 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-11T15:19:11.803 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi134.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-11T15:32:32.270 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2023-08-11T15:32:32.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-08-11T15:32:32.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:32:32.384 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-08-11T15:32:32.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-08-11T15:32:32.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-08-11T15:32:32.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:32:32.567 DEBUG:teuthology.orchestra.run.smithi134:> true 2023-08-11T15:32:32.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-08-11T15:32:32.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-08-11T15:32:32.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T15:32:32.704 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-08-11T15:32:32.770 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-08-11T15:32:32.771 INFO:teuthology.run_tasks:Running task clock... 2023-08-11T15:32:32.781 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-11T15:32:32.782 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T15:32:32.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T15:32:32.788 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T15:32:32.788 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T15:32:32.790 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T15:32:32.791 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T15:32:32.823 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T15:32:32.839 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T15:32:32.846 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T15:32:32.862 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T15:32:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T15:32:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T15:32:32.913 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2023-08-11T15:32:32.925 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2023-08-11T15:32:32.940 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2023-08-11T15:32:32.941 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T15:32:32.952 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2023-08-11T15:32:32.955 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T15:32:32.967 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T15:32:32.981 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T15:32:33.000 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2023-08-11T15:32:33.030 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2023-08-11T15:32:33.034 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-08-11T15:32:33.039 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T15:32:33.039 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-08-11T15:32:33.039 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.039 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.039 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.040 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.051 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2023-08-11T15:32:33.056 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T15:32:33.056 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-08-11T15:32:33.057 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.057 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.057 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.057 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T15:32:33.102 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T15:32:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-08-11T15:32:33.209 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T15:32:33.210 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-08-11T15:32:33.210 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.210 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.210 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.210 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T15:32:33.211 INFO:teuthology.run_tasks:Running task pexec... 2023-08-11T15:32:33.221 INFO:teuthology.task.pexec:Executing custom commands... 2023-08-11T15:32:33.221 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:32:33.222 DEBUG:teuthology.orchestra.run.smithi032:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-11T15:32:33.222 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi134.front.sepia.ceph.com 2023-08-11T15:32:33.222 DEBUG:teuthology.orchestra.run.smithi134:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-11T15:32:33.223 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi192.front.sepia.ceph.com 2023-08-11T15:32:33.223 DEBUG:teuthology.orchestra.run.smithi192:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-11T15:32:33.593 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-08-11T15:32:33.594 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-08-11T15:32:33.736 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-08-11T15:32:35.351 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:05:49 ago on Fri 11 Aug 2023 03:26:45 PM UTC. 2023-08-11T15:32:35.466 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:05:49 ago on Fri 11 Aug 2023 03:26:46 PM UTC. 2023-08-11T15:32:35.637 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:03 ago on Fri 11 Aug 2023 03:30:32 PM UTC. 2023-08-11T15:32:37.076 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-08-11T15:32:37.076 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-08-11T15:32:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2023-08-11T15:32:37.076 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-08-11T15:32:37.077 INFO:teuthology.orchestra.run.smithi192.stdout:Resetting modules: 2023-08-11T15:32:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools 2023-08-11T15:32:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:32:37.077 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-08-11T15:32:37.077 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-08-11T15:32:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:32:37.335 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-08-11T15:32:37.336 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-08-11T15:32:37.336 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2023-08-11T15:32:37.336 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-08-11T15:32:37.336 INFO:teuthology.orchestra.run.smithi134.stdout:Resetting modules: 2023-08-11T15:32:37.336 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools 2023-08-11T15:32:37.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:32:37.337 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-08-11T15:32:37.337 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-08-11T15:32:37.337 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:32:37.558 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-08-11T15:32:37.560 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-08-11T15:32:37.560 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2023-08-11T15:32:37.560 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-08-11T15:32:37.560 INFO:teuthology.orchestra.run.smithi032.stdout:Resetting modules: 2023-08-11T15:32:37.560 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools 2023-08-11T15:32:37.561 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:32:37.561 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-08-11T15:32:37.561 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-08-11T15:32:37.561 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:32:38.044 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:05:52 ago on Fri 11 Aug 2023 03:26:45 PM UTC. 2023-08-11T15:32:38.307 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:05:51 ago on Fri 11 Aug 2023 03:26:46 PM UTC. 2023-08-11T15:32:38.815 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:06 ago on Fri 11 Aug 2023 03:30:32 PM UTC. 2023-08-11T15:32:42.602 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-08-11T15:32:43.087 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-08-11T15:32:43.101 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-08-11T15:32:43.130 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-08-11T15:32:43.564 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-08-11T15:32:43.742 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-08-11T15:32:43.968 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:05:57 ago on Fri 11 Aug 2023 03:26:46 PM UTC. 2023-08-11T15:32:45.129 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:05:59 ago on Fri 11 Aug 2023 03:26:45 PM UTC. 2023-08-11T15:32:45.516 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:12 ago on Fri 11 Aug 2023 03:30:32 PM UTC. 2023-08-11T15:32:45.668 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-08-11T15:32:45.760 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-08-11T15:32:45.760 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-08-11T15:32:45.761 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-08-11T15:32:45.761 INFO:teuthology.orchestra.run.smithi134.stdout:Installing group/module packages: 2023-08-11T15:32:45.761 INFO:teuthology.orchestra.run.smithi134.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-11T15:32:45.761 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-11T15:32:45.761 INFO:teuthology.orchestra.run.smithi134.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-11T15:32:45.761 INFO:teuthology.orchestra.run.smithi134.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-11T15:32:45.762 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-11T15:32:45.762 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-11T15:32:45.762 INFO:teuthology.orchestra.run.smithi134.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-11T15:32:45.762 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-08-11T15:32:45.762 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-11T15:32:45.762 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-11T15:32:45.762 INFO:teuthology.orchestra.run.smithi134.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-11T15:32:45.763 INFO:teuthology.orchestra.run.smithi134.stdout:Installing module profiles: 2023-08-11T15:32:45.763 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools/common 2023-08-11T15:32:45.763 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:32:45.763 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2023-08-11T15:32:45.763 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools rhel8 2023-08-11T15:32:45.763 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:32:45.763 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-08-11T15:32:45.764 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-08-11T15:32:45.764 INFO:teuthology.orchestra.run.smithi134.stdout:Install 10 Packages 2023-08-11T15:32:45.764 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:32:45.765 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 18 M 2023-08-11T15:32:45.765 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 65 M 2023-08-11T15:32:45.765 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-08-11T15:32:45.904 INFO:teuthology.orchestra.run.smithi134.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 663 kB/s | 94 kB 00:00 2023-08-11T15:32:45.980 INFO:teuthology.orchestra.run.smithi134.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 188 kB/s | 41 kB 00:00 2023-08-11T15:32:46.013 INFO:teuthology.orchestra.run.smithi134.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 99 kB/s | 25 kB 00:00 2023-08-11T15:32:46.072 INFO:teuthology.orchestra.run.smithi134.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-08-11T15:32:46.130 INFO:teuthology.orchestra.run.smithi134.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 219 kB/s | 49 kB 00:00 2023-08-11T15:32:46.264 INFO:teuthology.orchestra.run.smithi134.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 778 kB/s | 149 kB 00:00 2023-08-11T15:32:46.298 INFO:teuthology.orchestra.run.smithi134.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.2 MB/s | 213 kB 00:00 2023-08-11T15:32:46.457 INFO:teuthology.orchestra.run.smithi134.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 18 MB/s | 8.0 MB 00:00 2023-08-11T15:32:46.523 INFO:teuthology.orchestra.run.smithi134.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.7 MB/s | 2.2 MB 00:00 2023-08-11T15:32:46.625 INFO:teuthology.orchestra.run.smithi134.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-08-11T15:32:46.626 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:32:46.626 INFO:teuthology.orchestra.run.smithi134.stdout:Total 21 MB/s | 18 MB 00:00 2023-08-11T15:32:46.823 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-08-11T15:32:46.842 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-08-11T15:32:46.843 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-08-11T15:32:46.993 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-08-11T15:32:47.025 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-08-11T15:32:47.029 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-08-11T15:32:47.086 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================================ 2023-08-11T15:32:47.086 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-08-11T15:32:47.086 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================================ 2023-08-11T15:32:47.086 INFO:teuthology.orchestra.run.smithi192.stdout:Installing group/module packages: 2023-08-11T15:32:47.086 INFO:teuthology.orchestra.run.smithi192.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-11T15:32:47.087 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-11T15:32:47.087 INFO:teuthology.orchestra.run.smithi192.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-11T15:32:47.087 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-11T15:32:47.087 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-11T15:32:47.087 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-11T15:32:47.087 INFO:teuthology.orchestra.run.smithi192.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-11T15:32:47.087 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-08-11T15:32:47.088 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-11T15:32:47.088 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-11T15:32:47.088 INFO:teuthology.orchestra.run.smithi192.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-11T15:32:47.088 INFO:teuthology.orchestra.run.smithi192.stdout:Installing module profiles: 2023-08-11T15:32:47.088 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools/common 2023-08-11T15:32:47.088 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:32:47.088 INFO:teuthology.orchestra.run.smithi192.stdout:Enabling module streams: 2023-08-11T15:32:47.089 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools rhel8 2023-08-11T15:32:47.090 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:32:47.090 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-08-11T15:32:47.090 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================================ 2023-08-11T15:32:47.091 INFO:teuthology.orchestra.run.smithi192.stdout:Install 10 Packages 2023-08-11T15:32:47.091 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:32:47.093 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 18 M 2023-08-11T15:32:47.093 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 65 M 2023-08-11T15:32:47.093 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-08-11T15:32:47.209 INFO:teuthology.orchestra.run.smithi192.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 780 kB/s | 94 kB 00:00 2023-08-11T15:32:47.250 INFO:teuthology.orchestra.run.smithi192.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 153 kB/s | 25 kB 00:00 2023-08-11T15:32:47.284 INFO:teuthology.orchestra.run.smithi192.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 208 kB/s | 41 kB 00:00 2023-08-11T15:32:47.301 INFO:teuthology.orchestra.run.smithi192.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 536 kB/s | 49 kB 00:00 2023-08-11T15:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.1 MB/s | 524 kB 00:00 2023-08-11T15:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.9 MB/s | 149 kB 00:00 2023-08-11T15:32:47.427 INFO:teuthology.orchestra.run.smithi192.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-08-11T15:32:47.527 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-08-11T15:32:47.549 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-08-11T15:32:47.585 INFO:teuthology.orchestra.run.smithi192.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 26 MB/s | 8.0 MB 00:00 2023-08-11T15:32:47.655 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================================ 2023-08-11T15:32:47.655 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2023-08-11T15:32:47.656 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================================ 2023-08-11T15:32:47.656 INFO:teuthology.orchestra.run.smithi032.stdout:Installing group/module packages: 2023-08-11T15:32:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-11T15:32:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-11T15:32:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-11T15:32:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-11T15:32:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout:Installing module profiles: 2023-08-11T15:32:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools/common 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling module streams: 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools rhel8 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================================ 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout:Install 10 Packages 2023-08-11T15:32:47.658 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:32:47.659 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 18 M 2023-08-11T15:32:47.659 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 65 M 2023-08-11T15:32:47.659 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-08-11T15:32:47.669 INFO:teuthology.orchestra.run.smithi192.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.1 MB/s | 2.2 MB 00:00 2023-08-11T15:32:47.723 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-11T15:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-08-11T15:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout:Total 27 MB/s | 18 MB 00:00 2023-08-11T15:32:47.795 INFO:teuthology.orchestra.run.smithi032.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 301 kB/s | 41 kB 00:00 2023-08-11T15:32:47.845 INFO:teuthology.orchestra.run.smithi032.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 505 kB/s | 94 kB 00:00 2023-08-11T15:32:47.870 INFO:teuthology.orchestra.run.smithi032.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 117 kB/s | 25 kB 00:00 2023-08-11T15:32:47.895 INFO:teuthology.orchestra.run.smithi032.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 492 kB/s | 49 kB 00:00 2023-08-11T15:32:47.897 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T15:32:47.963 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-08-11T15:32:47.981 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-08-11T15:32:47.982 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-08-11T15:32:48.012 INFO:teuthology.orchestra.run.smithi032.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.1 MB/s | 524 kB 00:00 2023-08-11T15:32:48.096 INFO:teuthology.orchestra.run.smithi032.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 746 kB/s | 149 kB 00:00 2023-08-11T15:32:48.116 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-11T15:32:48.146 INFO:teuthology.orchestra.run.smithi032.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.6 MB/s | 213 kB 00:00 2023-08-11T15:32:48.163 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-08-11T15:32:48.166 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-08-11T15:32:48.320 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-11T15:32:48.472 INFO:teuthology.orchestra.run.smithi032.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 13 MB/s | 8.0 MB 00:00 2023-08-11T15:32:48.530 INFO:teuthology.orchestra.run.smithi032.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 5.7 MB/s | 2.2 MB 00:00 2023-08-11T15:32:48.672 INFO:teuthology.orchestra.run.smithi032.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 12 MB/s | 7.0 MB 00:00 2023-08-11T15:32:48.673 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:32:48.673 INFO:teuthology.orchestra.run.smithi032.stdout:Total 18 MB/s | 18 MB 00:01 2023-08-11T15:32:48.677 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-11T15:32:48.835 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-08-11T15:32:49.017 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-08-11T15:32:49.036 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-11T15:32:49.041 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-08-11T15:32:49.041 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-08-11T15:32:49.207 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T15:32:49.400 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-11T15:32:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-11T15:32:49.555 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-08-11T15:32:49.560 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-08-11T15:32:49.621 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-11T15:32:49.969 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-11T15:32:50.122 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-08-11T15:32:50.328 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-11T15:32:50.328 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-11T15:32:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T15:32:50.552 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-11T15:32:50.705 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-11T15:32:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-11T15:32:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-11T15:32:50.948 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T15:32:50.963 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-11T15:32:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T15:32:51.222 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-11T15:32:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T15:32:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-11T15:32:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-11T15:32:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-11T15:32:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-11T15:32:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-11T15:32:51.223 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-11T15:32:51.224 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-11T15:32:51.343 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-11T15:32:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-11T15:32:51.871 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-11T15:32:52.048 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-11T15:32:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-11T15:32:52.283 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T15:32:52.424 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-11T15:32:52.425 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:06:05 ago on Fri 11 Aug 2023 03:26:46 PM UTC. 2023-08-11T15:32:52.550 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T15:32:52.550 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-11T15:32:52.550 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T15:32:52.550 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-11T15:32:52.550 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-11T15:32:52.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-11T15:32:52.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-11T15:32:52.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-11T15:32:52.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-11T15:32:52.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-11T15:32:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-11T15:32:53.486 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-11T15:32:53.657 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-11T15:32:53.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-11T15:32:53.682 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:06:08 ago on Fri 11 Aug 2023 03:26:45 PM UTC. 2023-08-11T15:32:53.864 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T15:32:54.254 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T15:32:54.254 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-11T15:32:54.255 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T15:32:54.255 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-11T15:32:54.255 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-11T15:32:54.255 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-11T15:32:54.256 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-11T15:32:54.256 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-11T15:32:54.256 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-11T15:32:54.257 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-11T15:32:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-11T15:32:55.669 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:23 ago on Fri 11 Aug 2023 03:30:32 PM UTC. 2023-08-11T15:32:58.483 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-08-11T15:32:58.692 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:32:58.692 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-08-11T15:32:58.693 INFO:teuthology.orchestra.run.smithi134.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:32:58.693 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T15:32:58.693 INFO:teuthology.orchestra.run.smithi134.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:32:58.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-11T15:32:58.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-11T15:32:58.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-11T15:32:58.695 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:32:58.695 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:32:58.695 INFO:teuthology.orchestra.run.smithi134.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T15:32:58.695 INFO:teuthology.orchestra.run.smithi134.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-11T15:32:58.696 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:32:58.696 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-08-11T15:32:58.862 DEBUG:teuthology.parallel:result is None 2023-08-11T15:33:00.344 INFO:teuthology.orchestra.run.smithi192.stdout:Installed products updated. 2023-08-11T15:33:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:33:00.541 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-08-11T15:33:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T15:33:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-11T15:33:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-11T15:33:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-11T15:33:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T15:33:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-11T15:33:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:33:00.543 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-08-11T15:33:00.696 DEBUG:teuthology.parallel:result is None 2023-08-11T15:33:03.159 INFO:teuthology.orchestra.run.smithi032.stdout:Installed products updated. 2023-08-11T15:33:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:33:03.385 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-08-11T15:33:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T15:33:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-11T15:33:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:33:03.387 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-08-11T15:33:03.594 DEBUG:teuthology.parallel:result is None 2023-08-11T15:33:03.595 INFO:teuthology.run_tasks:Running task install... 2023-08-11T15:33:03.607 DEBUG:teuthology.task.install:project ceph 2023-08-11T15:33:03.607 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'}} 2023-08-11T15:33:03.608 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'} 2023-08-11T15:33:03.608 INFO:teuthology.task.install:Using flavor: default 2023-08-11T15:33:03.613 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-11T15:33:03.614 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-11T15:33:03.614 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T15:33:03.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:33:03.617 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T15:33:03.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:33:03.618 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T15:33:03.619 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:33:04.279 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/ 2023-08-11T15:33:04.279 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T15:33:04.896 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T15:33:04.897 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:33:04.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T15:33:04.973 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T15:33:04.974 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T15:33:05.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T15:33:05.185 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T15:33:05.254 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2023-08-11T15:33:05.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2023-08-11T15:33:05.740 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-08-11T15:33:06.725 INFO:teuthology.orchestra.run.smithi032.stdout:44 files removed 2023-08-11T15:33:06.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T15:33:07.271 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-08-11T15:33:08.398 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/ 2023-08-11T15:33:08.398 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T15:33:10.041 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T15:33:10.041 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:33:10.042 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T15:33:10.074 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T15:33:10.075 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T15:33:10.146 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T15:33:10.226 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T15:33:10.294 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2023-08-11T15:33:10.295 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2023-08-11T15:33:10.667 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-08-11T15:33:11.570 INFO:teuthology.orchestra.run.smithi134.stdout:44 files removed 2023-08-11T15:33:11.623 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T15:33:11.955 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-08-11T15:33:12.035 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/ 2023-08-11T15:33:12.035 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T15:33:14.134 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T15:33:14.135 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:33:14.135 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T15:33:14.170 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T15:33:14.170 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T15:33:14.242 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T15:33:14.287 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 14 kB/s | 81 kB 00:05 2023-08-11T15:33:14.322 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T15:33:14.389 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2023-08-11T15:33:14.390 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2023-08-11T15:33:14.764 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-08-11T15:33:15.670 INFO:teuthology.orchestra.run.smithi192.stdout:44 files removed 2023-08-11T15:33:15.719 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T15:33:16.051 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-08-11T15:33:18.673 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 15 kB/s | 81 kB 00:05 2023-08-11T15:33:18.725 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 2.1 kB/s | 9.4 kB 00:04 2023-08-11T15:33:21.383 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 3.5 kB/s | 9.4 kB 00:02 2023-08-11T15:33:24.949 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 429 B/s | 1.5 kB 00:03 2023-08-11T15:33:25.100 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-08-11T15:33:26.319 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 8.8 kB/s | 81 kB 00:09 2023-08-11T15:33:29.728 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 16 MB 00:04 2023-08-11T15:33:33.192 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 509 kB/s | 24 kB 00:00 2023-08-11T15:33:34.377 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-08-11T15:33:39.110 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 756 B/s | 9.4 kB 00:12 2023-08-11T15:33:40.727 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 69 B/s | 1.5 kB 00:21 2023-08-11T15:33:40.877 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-08-11T15:33:42.452 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-08-11T15:33:42.628 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-08-11T15:33:43.084 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 384 B/s | 1.5 kB 00:03 2023-08-11T15:33:43.268 INFO:teuthology.orchestra.run.smithi192.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-08-11T15:33:45.454 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 16 MB 00:02 2023-08-11T15:33:46.711 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 386 kB/s | 24 kB 00:00 2023-08-11T15:33:48.138 INFO:teuthology.orchestra.run.smithi032.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 MB/s | 53 MB 00:01 2023-08-11T15:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 500 kB/s | 24 kB 00:00 2023-08-11T15:33:50.046 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-08-11T15:33:53.758 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T15:33:53.759 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T15:33:53.773 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-11T15:33:53.774 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-11T15:33:53.776 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-11T15:33:53.777 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-11T15:33:53.779 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-11T15:33:53.782 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-11T15:33:53.783 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-11T15:33:53.915 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-08-11T15:33:53.927 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-08-11T15:33:53.927 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-08-11T15:33:53.927 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-08-11T15:33:53.928 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2023-08-11T15:33:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-11T15:33:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-11T15:33:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.4 k 2023-08-11T15:33:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el8 ceph 5.4 M 2023-08-11T15:33:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 860 k 2023-08-11T15:33:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el8 ceph 167 k 2023-08-11T15:33:53.929 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el8 ceph 1.6 M 2023-08-11T15:33:53.929 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 131 k 2023-08-11T15:33:53.929 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 1.7 M 2023-08-11T15:33:53.929 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 7.4 M 2023-08-11T15:33:53.929 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 47 k 2023-08-11T15:33:53.929 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 14 M 2023-08-11T15:33:53.929 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el8 ceph 48 M 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 261 k 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 170 k 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 261 k 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 18 k 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-11T15:33:53.930 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 31 k 2023-08-11T15:33:53.931 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 783 k 2023-08-11T15:33:53.931 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T15:33:53.931 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-11T15:33:53.931 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-11T15:33:53.931 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-11T15:33:53.931 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-11T15:33:53.932 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-11T15:33:53.932 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-11T15:33:53.932 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el8 ceph 197 k 2023-08-11T15:33:53.932 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el8 ceph 370 k 2023-08-11T15:33:53.932 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 364 k 2023-08-11T15:33:53.932 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 107 k 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 94 k 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.5 M 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 179 k 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.7 M 2023-08-11T15:33:53.933 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.6 M 2023-08-11T15:33:53.934 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-08-11T15:33:53.934 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-11T15:33:53.934 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 22 M 2023-08-11T15:33:53.935 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 23 k 2023-08-11T15:33:53.935 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el8 ceph 2.3 M 2023-08-11T15:33:53.935 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 241 k 2023-08-11T15:33:53.935 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el8 ceph 4.6 M 2023-08-11T15:33:53.935 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el8 ceph 18 M 2023-08-11T15:33:53.936 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 15 k 2023-08-11T15:33:53.936 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el8 ceph 24 k 2023-08-11T15:33:53.936 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-11T15:33:53.936 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-11T15:33:53.936 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-11T15:33:53.936 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-11T15:33:53.936 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-11T15:33:53.937 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 473 k 2023-08-11T15:33:53.938 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-11T15:33:53.938 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.8 M 2023-08-11T15:33:53.938 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-11T15:33:53.938 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-11T15:33:53.938 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-11T15:33:53.938 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-11T15:33:53.938 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-11T15:33:53.939 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-11T15:33:53.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-11T15:33:53.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-11T15:33:53.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-11T15:33:53.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el8 ceph 45 k 2023-08-11T15:33:53.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 125 k 2023-08-11T15:33:53.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-11T15:33:53.940 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-11T15:33:53.941 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-11T15:33:53.942 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-11T15:33:53.943 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-11T15:33:53.944 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-11T15:33:53.944 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-11T15:33:53.944 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-11T15:33:53.944 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-11T15:33:53.944 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T15:33:53.944 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-11T15:33:53.944 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-11T15:33:53.945 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-11T15:33:53.946 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-11T15:33:53.947 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2023-08-11T15:33:53.947 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql 10 2023-08-11T15:33:53.947 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:33:53.947 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-08-11T15:33:53.947 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-08-11T15:33:53.947 INFO:teuthology.orchestra.run.smithi134.stdout:Install 128 Packages 2023-08-11T15:33:53.947 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 3 Packages 2023-08-11T15:33:53.948 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:33:53.948 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 206 M 2023-08-11T15:33:53.948 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-08-11T15:33:54.178 INFO:teuthology.orchestra.run.smithi134.stdout:(1/131): ceph-18.0.0-5437.g92684807.el8.x86_64. 32 kB/s | 6.4 kB 00:00 2023-08-11T15:33:55.353 INFO:teuthology.orchestra.run.smithi134.stdout:(2/131): ceph-fuse-18.0.0-5437.g92684807.el8.x8 732 kB/s | 860 kB 00:01 2023-08-11T15:33:55.445 INFO:teuthology.orchestra.run.smithi134.stdout:(3/131): ceph-base-18.0.0-5437.g92684807.el8.x8 3.7 MB/s | 5.4 MB 00:01 2023-08-11T15:33:55.545 INFO:teuthology.orchestra.run.smithi134.stdout:(4/131): ceph-immutable-object-cache-18.0.0-543 872 kB/s | 167 kB 00:00 2023-08-11T15:33:55.712 INFO:teuthology.orchestra.run.smithi134.stdout:(5/131): ceph-mds-18.0.0-5437.g92684807.el8.x86 8.6 MB/s | 2.3 MB 00:00 2023-08-11T15:33:56.171 INFO:teuthology.orchestra.run.smithi134.stdout:(6/131): ceph-mon-18.0.0-5437.g92684807.el8.x86 10 MB/s | 4.6 MB 00:00 2023-08-11T15:33:56.754 INFO:teuthology.orchestra.run.smithi134.stdout:(7/131): ceph-common-18.0.0-5437.g92684807.el8. 8.0 MB/s | 22 MB 00:02 2023-08-11T15:33:56.788 INFO:teuthology.orchestra.run.smithi134.stdout:(8/131): ceph-mgr-18.0.0-5437.g92684807.el8.x86 1.3 MB/s | 1.6 MB 00:01 2023-08-11T15:33:56.855 INFO:teuthology.orchestra.run.smithi134.stdout:(9/131): ceph-selinux-18.0.0-5437.g92684807.el8 366 kB/s | 24 kB 00:00 2023-08-11T15:33:57.755 INFO:teuthology.orchestra.run.smithi134.stdout:(10/131): ceph-osd-18.0.0-5437.g92684807.el8.x8 11 MB/s | 18 MB 00:01 2023-08-11T15:33:57.806 INFO:teuthology.orchestra.run.smithi032.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 MB/s | 47 MB 00:01 2023-08-11T15:33:57.930 INFO:teuthology.orchestra.run.smithi134.stdout:(11/131): ceph-radosgw-18.0.0-5437.g92684807.el 12 MB/s | 14 MB 00:01 2023-08-11T15:33:57.955 INFO:teuthology.orchestra.run.smithi134.stdout:(12/131): cephfs-mirror-18.0.0-5437.g92684807.e 1.3 MB/s | 261 kB 00:00 2023-08-11T15:33:57.981 INFO:teuthology.orchestra.run.smithi134.stdout:(13/131): libcephfs-devel-18.0.0-5437.g92684807 623 kB/s | 31 kB 00:00 2023-08-11T15:33:58.031 INFO:teuthology.orchestra.run.smithi134.stdout:(14/131): libcephfs2-18.0.0-5437.g92684807.el8. 10 MB/s | 783 kB 00:00 2023-08-11T15:33:58.055 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 53 MB 00:01 2023-08-11T15:33:58.056 INFO:teuthology.orchestra.run.smithi134.stdout:(15/131): libcephsqlite-18.0.0-5437.g92684807.e 1.8 MB/s | 139 kB 00:00 2023-08-11T15:33:58.073 INFO:teuthology.orchestra.run.smithi134.stdout:(16/131): librados-devel-18.0.0-5437.g92684807. 3.2 MB/s | 139 kB 00:00 2023-08-11T15:33:58.115 INFO:teuthology.orchestra.run.smithi134.stdout:(17/131): libradosstriper1-18.0.0-5437.g9268480 7.9 MB/s | 473 kB 00:00 2023-08-11T15:33:58.182 INFO:teuthology.orchestra.run.smithi134.stdout:(18/131): python3-ceph-argparse-18.0.0-5437.g92 670 kB/s | 45 kB 00:00 2023-08-11T15:33:58.249 INFO:teuthology.orchestra.run.smithi134.stdout:(19/131): python3-ceph-common-18.0.0-5437.g9268 1.8 MB/s | 125 kB 00:00 2023-08-11T15:33:58.316 INFO:teuthology.orchestra.run.smithi134.stdout:(20/131): python3-cephfs-18.0.0-5437.g92684807. 2.9 MB/s | 197 kB 00:00 2023-08-11T15:33:58.383 INFO:teuthology.orchestra.run.smithi134.stdout:(21/131): python3-rados-18.0.0-5437.g92684807.e 5.5 MB/s | 370 kB 00:00 2023-08-11T15:33:58.500 INFO:teuthology.orchestra.run.smithi134.stdout:(22/131): librgw2-18.0.0-5437.g92684807.el8.x86 16 MB/s | 6.8 MB 00:00 2023-08-11T15:33:58.525 INFO:teuthology.orchestra.run.smithi134.stdout:(23/131): python3-rbd-18.0.0-5437.g92684807.el8 2.5 MB/s | 364 kB 00:00 2023-08-11T15:33:58.551 INFO:teuthology.orchestra.run.smithi134.stdout:(24/131): python3-rgw-18.0.0-5437.g92684807.el8 2.1 MB/s | 107 kB 00:00 2023-08-11T15:33:58.576 INFO:teuthology.orchestra.run.smithi134.stdout:(25/131): rbd-fuse-18.0.0-5437.g92684807.el8.x8 1.8 MB/s | 94 kB 00:00 2023-08-11T15:33:58.643 INFO:teuthology.orchestra.run.smithi134.stdout:(26/131): rbd-nbd-18.0.0-5437.g92684807.el8.x86 2.6 MB/s | 179 kB 00:00 2023-08-11T15:33:58.693 INFO:teuthology.orchestra.run.smithi134.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5437.g 462 kB/s | 23 kB 00:00 2023-08-11T15:33:58.769 INFO:teuthology.orchestra.run.smithi134.stdout:(28/131): rbd-mirror-18.0.0-5437.g92684807.el8. 16 MB/s | 3.5 MB 00:00 2023-08-11T15:33:58.794 INFO:teuthology.orchestra.run.smithi134.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5437.g9268480 1.3 MB/s | 131 kB 00:00 2023-08-11T15:33:58.919 INFO:teuthology.orchestra.run.smithi134.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5437.g92684 12 MB/s | 1.7 MB 00:00 2023-08-11T15:33:58.995 INFO:teuthology.orchestra.run.smithi134.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5437.g92 3.2 MB/s | 241 kB 00:00 2023-08-11T15:33:59.062 INFO:teuthology.orchestra.run.smithi134.stdout:(32/131): ceph-mgr-rook-18.0.0-5437.g92684807.e 711 kB/s | 47 kB 00:00 2023-08-11T15:33:59.120 INFO:teuthology.orchestra.run.smithi134.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5437.g9 250 kB/s | 15 kB 00:00 2023-08-11T15:33:59.187 INFO:teuthology.orchestra.run.smithi134.stdout:(34/131): ceph-volume-18.0.0-5437.g92684807.el8 3.8 MB/s | 261 kB 00:00 2023-08-11T15:33:59.296 INFO:teuthology.orchestra.run.smithi134.stdout:(35/131): ceph-mgr-diskprediction-local-18.0.0- 15 MB/s | 7.4 MB 00:00 2023-08-11T15:33:59.321 INFO:teuthology.orchestra.run.smithi134.stdout:(36/131): cephadm-18.0.0-5437.g92684807.el8.noa 1.2 MB/s | 170 kB 00:00 2023-08-11T15:33:59.346 INFO:teuthology.orchestra.run.smithi134.stdout:(37/131): cephfs-top-18.0.0-5437.g92684807.el8. 359 kB/s | 18 kB 00:00 2023-08-11T15:33:59.471 INFO:teuthology.orchestra.run.smithi134.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 471 kB/s | 59 kB 00:00 2023-08-11T15:33:59.538 INFO:teuthology.orchestra.run.smithi134.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-08-11T15:33:59.556 INFO:teuthology.orchestra.run.smithi134.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 899 kB/s | 75 kB 00:00 2023-08-11T15:33:59.631 INFO:teuthology.orchestra.run.smithi134.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 582 kB/s | 44 kB 00:00 2023-08-11T15:33:59.656 INFO:teuthology.orchestra.run.smithi134.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2023-08-11T15:33:59.706 INFO:teuthology.orchestra.run.smithi134.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-08-11T15:33:59.731 INFO:teuthology.orchestra.run.smithi134.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 439 kB/s | 33 kB 00:00 2023-08-11T15:33:59.782 INFO:teuthology.orchestra.run.smithi134.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-08-11T15:33:59.832 INFO:teuthology.orchestra.run.smithi134.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-08-11T15:33:59.874 INFO:teuthology.orchestra.run.smithi134.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-08-11T15:33:59.916 INFO:teuthology.orchestra.run.smithi134.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2023-08-11T15:33:59.958 INFO:teuthology.orchestra.run.smithi134.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 421 kB/s | 18 kB 00:00 2023-08-11T15:34:00.034 INFO:teuthology.orchestra.run.smithi134.stdout:(50/131): python3-certifi-2018.10.15-7.el8.noar 50 kB/s | 16 kB 00:00 2023-08-11T15:34:00.068 INFO:teuthology.orchestra.run.smithi134.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-08-11T15:34:00.170 INFO:teuthology.orchestra.run.smithi134.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 359 kB/s | 49 kB 00:00 2023-08-11T15:34:00.234 INFO:teuthology.orchestra.run.smithi134.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 934 kB/s | 60 kB 00:00 2023-08-11T15:34:00.260 INFO:teuthology.orchestra.run.smithi134.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 305 kB/s | 59 kB 00:00 2023-08-11T15:34:00.318 INFO:teuthology.orchestra.run.smithi134.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-08-11T15:34:00.352 INFO:teuthology.orchestra.run.smithi134.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-08-11T15:34:00.386 INFO:teuthology.orchestra.run.smithi134.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 501 kB/s | 34 kB 00:00 2023-08-11T15:34:00.411 INFO:teuthology.orchestra.run.smithi134.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-08-11T15:34:00.436 INFO:teuthology.orchestra.run.smithi134.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-08-11T15:34:00.461 INFO:teuthology.orchestra.run.smithi134.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-08-11T15:34:00.486 INFO:teuthology.orchestra.run.smithi134.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-08-11T15:34:00.512 INFO:teuthology.orchestra.run.smithi134.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-08-11T15:34:00.537 INFO:teuthology.orchestra.run.smithi134.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 547 kB/s | 27 kB 00:00 2023-08-11T15:34:00.562 INFO:teuthology.orchestra.run.smithi134.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-08-11T15:34:00.604 INFO:teuthology.orchestra.run.smithi134.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-08-11T15:34:00.630 INFO:teuthology.orchestra.run.smithi134.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-08-11T15:34:00.655 INFO:teuthology.orchestra.run.smithi134.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-08-11T15:34:00.680 INFO:teuthology.orchestra.run.smithi134.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2023-08-11T15:34:00.714 INFO:teuthology.orchestra.run.smithi134.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 70 kB 00:00 2023-08-11T15:34:00.781 INFO:teuthology.orchestra.run.smithi134.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-08-11T15:34:00.915 INFO:teuthology.orchestra.run.smithi134.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 863 kB/s | 110 kB 00:00 2023-08-11T15:34:01.048 INFO:teuthology.orchestra.run.smithi134.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 406 kB/s | 54 kB 00:00 2023-08-11T15:34:01.232 INFO:teuthology.orchestra.run.smithi134.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-08-11T15:34:01.257 INFO:teuthology.orchestra.run.smithi134.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2023-08-11T15:34:01.674 INFO:teuthology.orchestra.run.smithi134.stdout:(75/131): ceph-test-18.0.0-5437.g92684807.el8.x 10 MB/s | 48 MB 00:04 2023-08-11T15:34:01.700 INFO:teuthology.orchestra.run.smithi134.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 88 kB/s | 39 kB 00:00 2023-08-11T15:34:01.733 INFO:teuthology.orchestra.run.smithi134.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 645 kB/s | 323 kB 00:00 2023-08-11T15:34:01.842 INFO:teuthology.orchestra.run.smithi134.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-08-11T15:34:01.875 INFO:teuthology.orchestra.run.smithi134.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2023-08-11T15:34:01.934 INFO:teuthology.orchestra.run.smithi134.stdout:(80/131): bison-3.0.4-10.el8.x86_64.rpm 2.6 MB/s | 688 kB 00:00 2023-08-11T15:34:01.960 INFO:teuthology.orchestra.run.smithi134.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-08-11T15:34:02.026 INFO:teuthology.orchestra.run.smithi134.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.6 MB/s | 713 kB 00:00 2023-08-11T15:34:02.085 INFO:teuthology.orchestra.run.smithi134.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 212 kB/s | 27 kB 00:00 2023-08-11T15:34:02.135 INFO:teuthology.orchestra.run.smithi134.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-08-11T15:34:02.202 INFO:teuthology.orchestra.run.smithi134.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-08-11T15:34:02.227 INFO:teuthology.orchestra.run.smithi134.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 309 kB/s | 44 kB 00:00 2023-08-11T15:34:02.353 INFO:teuthology.orchestra.run.smithi134.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-08-11T15:34:02.370 INFO:teuthology.orchestra.run.smithi134.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2023-08-11T15:34:02.478 INFO:teuthology.orchestra.run.smithi134.stdout:(89/131): python3-werkzeug-0.12.2-4.el8.noarch. 1.6 MB/s | 457 kB 00:00 2023-08-11T15:34:02.603 INFO:teuthology.orchestra.run.smithi134.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 255 kB/s | 64 kB 00:00 2023-08-11T15:34:02.620 INFO:teuthology.orchestra.run.smithi134.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2023-08-11T15:34:02.646 INFO:teuthology.orchestra.run.smithi134.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 458 kB/s | 126 kB 00:00 2023-08-11T15:34:02.729 INFO:teuthology.orchestra.run.smithi134.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 825 kB/s | 103 kB 00:00 2023-08-11T15:34:02.771 INFO:teuthology.orchestra.run.smithi134.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2023-08-11T15:34:02.846 INFO:teuthology.orchestra.run.smithi134.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.7 MB/s | 198 kB 00:00 2023-08-11T15:34:02.955 INFO:teuthology.orchestra.run.smithi134.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-08-11T15:34:02.980 INFO:teuthology.orchestra.run.smithi134.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2023-08-11T15:34:03.059 INFO:teuthology.orchestra.run.smithi134.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.3 MB/s | 354 kB 00:00 2023-08-11T15:34:03.156 INFO:teuthology.orchestra.run.smithi134.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-08-11T15:34:03.189 INFO:teuthology.orchestra.run.smithi134.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.1 MB/s | 808 kB 00:00 2023-08-11T15:34:03.281 INFO:teuthology.orchestra.run.smithi134.stdout:(101/131): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-08-11T15:34:03.356 INFO:teuthology.orchestra.run.smithi134.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 7.5 MB/s | 1.5 MB 00:00 2023-08-11T15:34:03.382 INFO:teuthology.orchestra.run.smithi134.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 244 kB/s | 47 kB 00:00 2023-08-11T15:34:03.441 INFO:teuthology.orchestra.run.smithi134.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2023-08-11T15:34:03.474 INFO:teuthology.orchestra.run.smithi134.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-08-11T15:34:03.500 INFO:teuthology.orchestra.run.smithi134.stdout:(106/131): python3-jwt-1.6.1-2.el8.noarch.rpm 298 kB/s | 43 kB 00:00 2023-08-11T15:34:03.567 INFO:teuthology.orchestra.run.smithi134.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 946 kB/s | 87 kB 00:00 2023-08-11T15:34:03.592 INFO:teuthology.orchestra.run.smithi134.stdout:(108/131): keyutils-libs-devel-1.5.10-9.el8.x86 318 kB/s | 48 kB 00:00 2023-08-11T15:34:03.642 INFO:teuthology.orchestra.run.smithi134.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 301 kB/s | 43 kB 00:00 2023-08-11T15:34:03.709 INFO:teuthology.orchestra.run.smithi134.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 768 kB/s | 109 kB 00:00 2023-08-11T15:34:03.743 INFO:teuthology.orchestra.run.smithi134.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.4 MB/s | 560 kB 00:00 2023-08-11T15:34:03.777 INFO:teuthology.orchestra.run.smithi134.stdout:(112/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2023-08-11T15:34:03.818 INFO:teuthology.orchestra.run.smithi134.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 172 kB/s | 19 kB 00:00 2023-08-11T15:34:03.877 INFO:teuthology.orchestra.run.smithi134.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-08-11T15:34:03.927 INFO:teuthology.orchestra.run.smithi134.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-08-11T15:34:04.003 INFO:teuthology.orchestra.run.smithi134.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-08-11T15:34:04.028 INFO:teuthology.orchestra.run.smithi134.stdout:(117/131): libverto-devel-0.3.0-5.el8.x86_64.rp 85 kB/s | 18 kB 00:00 2023-08-11T15:34:04.062 INFO:teuthology.orchestra.run.smithi134.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.7 MB/s | 238 kB 00:00 2023-08-11T15:34:04.120 INFO:teuthology.orchestra.run.smithi134.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 787 kB/s | 72 kB 00:00 2023-08-11T15:34:04.145 INFO:teuthology.orchestra.run.smithi134.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 993 kB/s | 83 kB 00:00 2023-08-11T15:34:04.219 INFO:teuthology.orchestra.run.smithi134.stdout:(121/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 11 MB/s | 2.3 MB 00:00 2023-08-11T15:34:04.246 INFO:teuthology.orchestra.run.smithi134.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 477 kB/s | 60 kB 00:00 2023-08-11T15:34:04.271 INFO:teuthology.orchestra.run.smithi134.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 782 kB/s | 98 kB 00:00 2023-08-11T15:34:04.313 INFO:teuthology.orchestra.run.smithi134.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 577 kB/s | 39 kB 00:00 2023-08-11T15:34:04.338 INFO:teuthology.orchestra.run.smithi134.stdout:(125/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 485 kB/s | 58 kB 00:00 2023-08-11T15:34:04.380 INFO:teuthology.orchestra.run.smithi134.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-08-11T15:34:04.406 INFO:teuthology.orchestra.run.smithi134.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.0 MB/s | 138 kB 00:00 2023-08-11T15:34:04.448 INFO:teuthology.orchestra.run.smithi134.stdout:(128/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.6 MB/s | 644 kB 00:00 2023-08-11T15:34:04.573 INFO:teuthology.orchestra.run.smithi134.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.2 MB/s | 798 kB 00:00 2023-08-11T15:34:04.748 INFO:teuthology.orchestra.run.smithi134.stdout:(130/131): librbd1-18.0.0-5437.g92684807.el8.x8 11 MB/s | 3.6 MB 00:00 2023-08-11T15:34:04.840 INFO:teuthology.orchestra.run.smithi134.stdout:(131/131): librados2-18.0.0-5437.g92684807.el8. 8.0 MB/s | 3.7 MB 00:00 2023-08-11T15:34:04.843 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:34:04.844 INFO:teuthology.orchestra.run.smithi134.stdout:Total 19 MB/s | 206 MB 00:10 2023-08-11T15:34:05.655 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-08-11T15:34:05.774 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-08-11T15:34:05.774 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-08-11T15:34:06.902 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-08-11T15:34:06.906 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-08-11T15:34:09.328 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T15:34:09.329 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T15:34:09.344 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-11T15:34:09.345 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-11T15:34:09.346 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-11T15:34:09.347 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-11T15:34:09.349 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-11T15:34:09.353 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-11T15:34:09.354 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-11T15:34:09.485 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-08-11T15:34:09.497 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-08-11T15:34:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-08-11T15:34:09.498 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-08-11T15:34:09.498 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2023-08-11T15:34:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-11T15:34:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-11T15:34:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.4 k 2023-08-11T15:34:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el8 ceph 5.4 M 2023-08-11T15:34:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 860 k 2023-08-11T15:34:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el8 ceph 167 k 2023-08-11T15:34:09.499 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el8 ceph 1.6 M 2023-08-11T15:34:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 131 k 2023-08-11T15:34:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 1.7 M 2023-08-11T15:34:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 7.4 M 2023-08-11T15:34:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 47 k 2023-08-11T15:34:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 14 M 2023-08-11T15:34:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el8 ceph 48 M 2023-08-11T15:34:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 261 k 2023-08-11T15:34:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 170 k 2023-08-11T15:34:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 261 k 2023-08-11T15:34:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 18 k 2023-08-11T15:34:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-11T15:34:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-11T15:34:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-11T15:34:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 31 k 2023-08-11T15:34:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 783 k 2023-08-11T15:34:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T15:34:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-11T15:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-11T15:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-11T15:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-11T15:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-11T15:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-11T15:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el8 ceph 197 k 2023-08-11T15:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el8 ceph 370 k 2023-08-11T15:34:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 364 k 2023-08-11T15:34:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 107 k 2023-08-11T15:34:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 94 k 2023-08-11T15:34:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.5 M 2023-08-11T15:34:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 179 k 2023-08-11T15:34:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-11T15:34:09.504 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2023-08-11T15:34:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-11T15:34:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.7 M 2023-08-11T15:34:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.6 M 2023-08-11T15:34:09.505 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-08-11T15:34:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-11T15:34:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 22 M 2023-08-11T15:34:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 23 k 2023-08-11T15:34:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el8 ceph 2.3 M 2023-08-11T15:34:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 241 k 2023-08-11T15:34:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el8 ceph 4.6 M 2023-08-11T15:34:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el8 ceph 18 M 2023-08-11T15:34:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 15 k 2023-08-11T15:34:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el8 ceph 24 k 2023-08-11T15:34:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-11T15:34:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-11T15:34:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-11T15:34:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-11T15:34:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-11T15:34:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 473 k 2023-08-11T15:34:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-11T15:34:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.8 M 2023-08-11T15:34:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-11T15:34:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-11T15:34:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-11T15:34:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-11T15:34:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-11T15:34:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-11T15:34:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-11T15:34:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-11T15:34:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-11T15:34:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el8 ceph 45 k 2023-08-11T15:34:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 125 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-11T15:34:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-11T15:34:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-11T15:34:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-11T15:34:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-11T15:34:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T15:34:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-11T15:34:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-11T15:34:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-11T15:34:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-11T15:34:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-11T15:34:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-11T15:34:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-11T15:34:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-11T15:34:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-11T15:34:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-11T15:34:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-11T15:34:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-11T15:34:09.518 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-11T15:34:09.519 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-11T15:34:09.519 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-11T15:34:09.519 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-11T15:34:09.519 INFO:teuthology.orchestra.run.smithi192.stdout:Enabling module streams: 2023-08-11T15:34:09.519 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql 10 2023-08-11T15:34:09.519 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:34:09.519 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-08-11T15:34:09.520 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-08-11T15:34:09.520 INFO:teuthology.orchestra.run.smithi192.stdout:Install 128 Packages 2023-08-11T15:34:09.520 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 3 Packages 2023-08-11T15:34:09.520 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:34:09.521 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 206 M 2023-08-11T15:34:09.521 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-08-11T15:34:09.664 INFO:teuthology.orchestra.run.smithi192.stdout:(1/131): ceph-18.0.0-5437.g92684807.el8.x86_64. 53 kB/s | 6.4 kB 00:00 2023-08-11T15:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout:(2/131): ceph-fuse-18.0.0-5437.g92684807.el8.x8 3.0 MB/s | 860 kB 00:00 2023-08-11T15:34:10.023 INFO:teuthology.orchestra.run.smithi192.stdout:(3/131): ceph-base-18.0.0-5437.g92684807.el8.x8 11 MB/s | 5.4 MB 00:00 2023-08-11T15:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:(4/131): ceph-immutable-object-cache-18.0.0-543 1.6 MB/s | 167 kB 00:00 2023-08-11T15:34:10.120 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T15:34:10.121 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T15:34:10.137 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-11T15:34:10.138 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-11T15:34:10.139 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-11T15:34:10.140 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-11T15:34:10.143 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-11T15:34:10.146 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-11T15:34:10.147 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-11T15:34:10.190 INFO:teuthology.orchestra.run.smithi192.stdout:(5/131): ceph-mds-18.0.0-5437.g92684807.el8.x86 14 MB/s | 2.3 MB 00:00 2023-08-11T15:34:10.299 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-08-11T15:34:10.313 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-08-11T15:34:10.314 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2023-08-11T15:34:10.314 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-08-11T15:34:10.315 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2023-08-11T15:34:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-11T15:34:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-11T15:34:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.4 k 2023-08-11T15:34:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el8 ceph 5.4 M 2023-08-11T15:34:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 860 k 2023-08-11T15:34:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el8 ceph 167 k 2023-08-11T15:34:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el8 ceph 1.6 M 2023-08-11T15:34:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 131 k 2023-08-11T15:34:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 1.7 M 2023-08-11T15:34:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 7.4 M 2023-08-11T15:34:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 47 k 2023-08-11T15:34:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 14 M 2023-08-11T15:34:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el8 ceph 48 M 2023-08-11T15:34:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 261 k 2023-08-11T15:34:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 170 k 2023-08-11T15:34:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 261 k 2023-08-11T15:34:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 18 k 2023-08-11T15:34:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-11T15:34:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-11T15:34:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-11T15:34:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 31 k 2023-08-11T15:34:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 783 k 2023-08-11T15:34:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T15:34:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-11T15:34:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-11T15:34:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-11T15:34:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-11T15:34:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-11T15:34:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-11T15:34:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el8 ceph 197 k 2023-08-11T15:34:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el8 ceph 370 k 2023-08-11T15:34:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 364 k 2023-08-11T15:34:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 107 k 2023-08-11T15:34:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 94 k 2023-08-11T15:34:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.5 M 2023-08-11T15:34:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 179 k 2023-08-11T15:34:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-11T15:34:10.322 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2023-08-11T15:34:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-11T15:34:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.7 M 2023-08-11T15:34:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.6 M 2023-08-11T15:34:10.322 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-08-11T15:34:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-11T15:34:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 22 M 2023-08-11T15:34:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 23 k 2023-08-11T15:34:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el8 ceph 2.3 M 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 241 k 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el8 ceph 4.6 M 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el8 ceph 18 M 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 15 k 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el8 ceph 24 k 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-11T15:34:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-11T15:34:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-11T15:34:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-11T15:34:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T15:34:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-11T15:34:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-11T15:34:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-11T15:34:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-11T15:34:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-11T15:34:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-11T15:34:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 473 k 2023-08-11T15:34:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-11T15:34:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.8 M 2023-08-11T15:34:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-11T15:34:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-11T15:34:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el8 ceph 45 k 2023-08-11T15:34:10.328 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 125 k 2023-08-11T15:34:10.329 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-11T15:34:10.329 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-11T15:34:10.329 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-11T15:34:10.329 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-11T15:34:10.329 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-11T15:34:10.329 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-11T15:34:10.329 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-11T15:34:10.330 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-11T15:34:10.331 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-11T15:34:10.332 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-11T15:34:10.332 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-11T15:34:10.332 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-11T15:34:10.332 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-11T15:34:10.332 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-11T15:34:10.332 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-11T15:34:10.332 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-11T15:34:10.333 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T15:34:10.333 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-11T15:34:10.333 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-11T15:34:10.333 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-11T15:34:10.334 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-11T15:34:10.335 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-11T15:34:10.335 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-11T15:34:10.335 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-11T15:34:10.335 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-11T15:34:10.335 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-11T15:34:10.335 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-11T15:34:10.335 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling module streams: 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql 10 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout:Install 128 Packages 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 3 Packages 2023-08-11T15:34:10.336 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:34:10.337 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 206 M 2023-08-11T15:34:10.337 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-08-11T15:34:10.407 INFO:teuthology.orchestra.run.smithi192.stdout:(6/131): ceph-mgr-18.0.0-5437.g92684807.el8.x86 4.4 MB/s | 1.6 MB 00:00 2023-08-11T15:34:10.482 INFO:teuthology.orchestra.run.smithi192.stdout:(7/131): ceph-mon-18.0.0-5437.g92684807.el8.x86 16 MB/s | 4.6 MB 00:00 2023-08-11T15:34:10.731 INFO:teuthology.orchestra.run.smithi032.stdout:(1/131): ceph-18.0.0-5437.g92684807.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-08-11T15:34:11.365 INFO:teuthology.orchestra.run.smithi192.stdout:(8/131): ceph-radosgw-18.0.0-5437.g92684807.el8 15 MB/s | 14 MB 00:00 2023-08-11T15:34:11.449 INFO:teuthology.orchestra.run.smithi192.stdout:(9/131): ceph-selinux-18.0.0-5437.g92684807.el8 293 kB/s | 24 kB 00:00 2023-08-11T15:34:11.456 INFO:teuthology.orchestra.run.smithi032.stdout:(2/131): ceph-fuse-18.0.0-5437.g92684807.el8.x8 1.2 MB/s | 860 kB 00:00 2023-08-11T15:34:11.640 INFO:teuthology.orchestra.run.smithi032.stdout:(3/131): ceph-immutable-object-cache-18.0.0-543 913 kB/s | 167 kB 00:00 2023-08-11T15:34:12.009 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-08-11T15:34:12.186 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-11T15:34:12.316 INFO:teuthology.orchestra.run.smithi192.stdout:(10/131): ceph-osd-18.0.0-5437.g92684807.el8.x8 9.3 MB/s | 18 MB 00:01 2023-08-11T15:34:12.364 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-11T15:34:12.391 INFO:teuthology.orchestra.run.smithi192.stdout:(11/131): cephfs-mirror-18.0.0-5437.g92684807.e 3.4 MB/s | 261 kB 00:00 2023-08-11T15:34:12.442 INFO:teuthology.orchestra.run.smithi032.stdout:(4/131): ceph-mds-18.0.0-5437.g92684807.el8.x86 2.9 MB/s | 2.3 MB 00:00 2023-08-11T15:34:12.449 INFO:teuthology.orchestra.run.smithi192.stdout:(12/131): libcephfs-devel-18.0.0-5437.g92684807 534 kB/s | 31 kB 00:00 2023-08-11T15:34:12.561 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-11T15:34:12.608 INFO:teuthology.orchestra.run.smithi192.stdout:(13/131): libcephfs2-18.0.0-5437.g92684807.el8. 4.8 MB/s | 783 kB 00:00 2023-08-11T15:34:12.683 INFO:teuthology.orchestra.run.smithi192.stdout:(14/131): libcephsqlite-18.0.0-5437.g92684807.e 1.8 MB/s | 139 kB 00:00 2023-08-11T15:34:12.730 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T15:34:12.767 INFO:teuthology.orchestra.run.smithi192.stdout:(15/131): librados-devel-18.0.0-5437.g92684807. 1.6 MB/s | 139 kB 00:00 2023-08-11T15:34:12.817 INFO:teuthology.orchestra.run.smithi032.stdout:(5/131): ceph-mgr-18.0.0-5437.g92684807.el8.x86 4.2 MB/s | 1.6 MB 00:00 2023-08-11T15:34:12.834 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T15:34:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:(16/131): libradosstriper1-18.0.0-5437.g9268480 5.5 MB/s | 473 kB 00:00 2023-08-11T15:34:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T15:34:13.492 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T15:34:13.684 INFO:teuthology.orchestra.run.smithi192.stdout:(17/131): librgw2-18.0.0-5437.g92684807.el8.x86 8.2 MB/s | 6.8 MB 00:00 2023-08-11T15:34:13.734 INFO:teuthology.orchestra.run.smithi032.stdout:(6/131): ceph-mon-18.0.0-5437.g92684807.el8.x86 5.0 MB/s | 4.6 MB 00:00 2023-08-11T15:34:13.743 INFO:teuthology.orchestra.run.smithi192.stdout:(18/131): python3-ceph-argparse-18.0.0-5437.g92 768 kB/s | 45 kB 00:00 2023-08-11T15:34:13.827 INFO:teuthology.orchestra.run.smithi192.stdout:(19/131): python3-ceph-common-18.0.0-5437.g9268 1.5 MB/s | 125 kB 00:00 2023-08-11T15:34:13.902 INFO:teuthology.orchestra.run.smithi192.stdout:(20/131): python3-cephfs-18.0.0-5437.g92684807. 2.6 MB/s | 197 kB 00:00 2023-08-11T15:34:13.994 INFO:teuthology.orchestra.run.smithi192.stdout:(21/131): python3-rados-18.0.0-5437.g92684807.e 3.9 MB/s | 370 kB 00:00 2023-08-11T15:34:14.039 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:34:14.086 INFO:teuthology.orchestra.run.smithi192.stdout:(22/131): python3-rbd-18.0.0-5437.g92684807.el8 3.9 MB/s | 364 kB 00:00 2023-08-11T15:34:14.169 INFO:teuthology.orchestra.run.smithi192.stdout:(23/131): python3-rgw-18.0.0-5437.g92684807.el8 1.2 MB/s | 107 kB 00:00 2023-08-11T15:34:14.253 INFO:teuthology.orchestra.run.smithi192.stdout:(24/131): rbd-fuse-18.0.0-5437.g92684807.el8.x8 1.1 MB/s | 94 kB 00:00 2023-08-11T15:34:14.632 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:34:14.795 INFO:teuthology.orchestra.run.smithi192.stdout:(25/131): rbd-mirror-18.0.0-5437.g92684807.el8. 6.5 MB/s | 3.5 MB 00:00 2023-08-11T15:34:14.813 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:34:14.879 INFO:teuthology.orchestra.run.smithi192.stdout:(26/131): rbd-nbd-18.0.0-5437.g92684807.el8.x86 2.1 MB/s | 179 kB 00:00 2023-08-11T15:34:14.941 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:34:14.954 INFO:teuthology.orchestra.run.smithi192.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5437.g 307 kB/s | 23 kB 00:00 2023-08-11T15:34:15.054 INFO:teuthology.orchestra.run.smithi192.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5437.g9268480 1.3 MB/s | 131 kB 00:00 2023-08-11T15:34:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:34:15.210 INFO:teuthology.orchestra.run.smithi032.stdout:(7/131): ceph-base-18.0.0-5437.g92684807.el8.x8 1.1 MB/s | 5.4 MB 00:04 2023-08-11T15:34:15.270 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:34:15.321 INFO:teuthology.orchestra.run.smithi192.stdout:(29/131): ceph-mgr-dashboard-18.0.0-5437.g92684 6.5 MB/s | 1.7 MB 00:00 2023-08-11T15:34:15.446 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T15:34:15.610 INFO:teuthology.orchestra.run.smithi032.stdout:(8/131): ceph-osd-18.0.0-5437.g92684807.el8.x86 9.5 MB/s | 18 MB 00:01 2023-08-11T15:34:15.674 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-11T15:34:15.685 INFO:teuthology.orchestra.run.smithi032.stdout:(9/131): ceph-selinux-18.0.0-5437.g92684807.el8 325 kB/s | 24 kB 00:00 2023-08-11T15:34:15.907 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-11T15:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout:(30/131): ceph-test-18.0.0-5437.g92684807.el8.x 10 MB/s | 48 MB 00:04 2023-08-11T15:34:16.247 INFO:teuthology.orchestra.run.smithi192.stdout:(31/131): ceph-mgr-diskprediction-local-18.0.0- 8.0 MB/s | 7.4 MB 00:00 2023-08-11T15:34:16.272 INFO:teuthology.orchestra.run.smithi192.stdout:(32/131): ceph-mgr-modules-core-18.0.0-5437.g92 1.8 MB/s | 241 kB 00:00 2023-08-11T15:34:16.297 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-11T15:34:16.305 INFO:teuthology.orchestra.run.smithi192.stdout:(33/131): ceph-mgr-rook-18.0.0-5437.g92684807.e 815 kB/s | 47 kB 00:00 2023-08-11T15:34:16.331 INFO:teuthology.orchestra.run.smithi192.stdout:(34/131): ceph-prometheus-alerts-18.0.0-5437.g9 248 kB/s | 15 kB 00:00 2023-08-11T15:34:16.373 INFO:teuthology.orchestra.run.smithi192.stdout:(35/131): ceph-volume-18.0.0-5437.g92684807.el8 3.8 MB/s | 261 kB 00:00 2023-08-11T15:34:16.448 INFO:teuthology.orchestra.run.smithi192.stdout:(36/131): cephadm-18.0.0-5437.g92684807.el8.noa 1.4 MB/s | 170 kB 00:00 2023-08-11T15:34:16.465 INFO:teuthology.orchestra.run.smithi192.stdout:(37/131): cephfs-top-18.0.0-5437.g92684807.el8. 195 kB/s | 18 kB 00:00 2023-08-11T15:34:16.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-11T15:34:16.704 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T15:34:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T15:34:16.979 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T15:34:17.040 INFO:teuthology.orchestra.run.smithi192.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 102 kB/s | 59 kB 00:00 2023-08-11T15:34:17.066 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T15:34:17.195 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T15:34:17.250 INFO:teuthology.orchestra.run.smithi192.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 360 kB/s | 75 kB 00:00 2023-08-11T15:34:17.299 INFO:teuthology.orchestra.run.smithi192.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 360 kB/s | 306 kB 00:00 2023-08-11T15:34:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T15:34:17.417 INFO:teuthology.orchestra.run.smithi192.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 372 kB/s | 44 kB 00:00 2023-08-11T15:34:17.529 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T15:34:17.708 INFO:teuthology.orchestra.run.smithi192.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 635 kB/s | 185 kB 00:00 2023-08-11T15:34:17.817 INFO:teuthology.orchestra.run.smithi192.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-08-11T15:34:17.926 INFO:teuthology.orchestra.run.smithi192.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-08-11T15:34:17.984 INFO:teuthology.orchestra.run.smithi192.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 647 kB/s | 476 kB 00:00 2023-08-11T15:34:18.147 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T15:34:18.218 INFO:teuthology.orchestra.run.smithi192.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 592 kB/s | 173 kB 00:00 2023-08-11T15:34:18.337 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T15:34:18.468 INFO:teuthology.orchestra.run.smithi192.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 325 kB/s | 81 kB 00:00 2023-08-11T15:34:18.660 INFO:teuthology.orchestra.run.smithi032.stdout:(10/131): ceph-test-18.0.0-5437.g92684807.el8.x 16 MB/s | 48 MB 00:02 2023-08-11T15:34:18.727 INFO:teuthology.orchestra.run.smithi192.stdout:(48/131): ceph-common-18.0.0-5437.g92684807.el8 2.4 MB/s | 22 MB 00:09 2023-08-11T15:34:18.728 INFO:teuthology.orchestra.run.smithi032.stdout:(11/131): cephfs-mirror-18.0.0-5437.g92684807.e 3.9 MB/s | 261 kB 00:00 2023-08-11T15:34:18.752 INFO:teuthology.orchestra.run.smithi192.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 39 kB/s | 11 kB 00:00 2023-08-11T15:34:18.769 INFO:teuthology.orchestra.run.smithi032.stdout:(12/131): libcephfs-devel-18.0.0-5437.g92684807 749 kB/s | 31 kB 00:00 2023-08-11T15:34:18.777 INFO:teuthology.orchestra.run.smithi192.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 485 kB/s | 384 kB 00:00 2023-08-11T15:34:18.845 INFO:teuthology.orchestra.run.smithi032.stdout:(13/131): libcephfs2-18.0.0-5437.g92684807.el8. 10 MB/s | 783 kB 00:00 2023-08-11T15:34:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 417 kB/s | 49 kB 00:00 2023-08-11T15:34:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:(14/131): libcephsqlite-18.0.0-5437.g92684807.e 2.7 MB/s | 139 kB 00:00 2023-08-11T15:34:18.930 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T15:34:18.945 INFO:teuthology.orchestra.run.smithi032.stdout:(15/131): librados-devel-18.0.0-5437.g92684807. 2.7 MB/s | 139 kB 00:00 2023-08-11T15:34:19.004 INFO:teuthology.orchestra.run.smithi032.stdout:(16/131): libradosstriper1-18.0.0-5437.g9268480 7.9 MB/s | 473 kB 00:00 2023-08-11T15:34:19.011 INFO:teuthology.orchestra.run.smithi192.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 504 kB/s | 59 kB 00:00 2023-08-11T15:34:19.128 INFO:teuthology.orchestra.run.smithi192.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 517 kB/s | 60 kB 00:00 2023-08-11T15:34:19.137 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T15:34:19.212 INFO:teuthology.orchestra.run.smithi192.stdout:(54/131): python3-jaraco-functools-2.0-4.el8.no 36 kB/s | 18 kB 00:00 2023-08-11T15:34:19.354 INFO:teuthology.orchestra.run.smithi192.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-08-11T15:34:19.429 INFO:teuthology.orchestra.run.smithi032.stdout:(17/131): librgw2-18.0.0-5437.g92684807.el8.x86 16 MB/s | 6.8 MB 00:00 2023-08-11T15:34:19.480 INFO:teuthology.orchestra.run.smithi032.stdout:(18/131): python3-ceph-argparse-18.0.0-5437.g92 900 kB/s | 45 kB 00:00 2023-08-11T15:34:19.530 INFO:teuthology.orchestra.run.smithi032.stdout:(19/131): python3-ceph-common-18.0.0-5437.g9268 2.5 MB/s | 125 kB 00:00 2023-08-11T15:34:19.554 INFO:teuthology.orchestra.run.smithi192.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 168 kB/s | 34 kB 00:00 2023-08-11T15:34:19.662 INFO:teuthology.orchestra.run.smithi192.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 530 kB/s | 283 kB 00:00 2023-08-11T15:34:19.747 INFO:teuthology.orchestra.run.smithi032.stdout:(20/131): python3-cephfs-18.0.0-5437.g92684807. 908 kB/s | 197 kB 00:00 2023-08-11T15:34:19.780 INFO:teuthology.orchestra.run.smithi192.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 546 kB/s | 64 kB 00:00 2023-08-11T15:34:19.806 INFO:teuthology.orchestra.run.smithi032.stdout:(21/131): python3-rados-18.0.0-5437.g92684807.e 6.2 MB/s | 370 kB 00:00 2023-08-11T15:34:19.808 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T15:34:19.864 INFO:teuthology.orchestra.run.smithi032.stdout:(22/131): python3-rbd-18.0.0-5437.g92684807.el8 6.1 MB/s | 364 kB 00:00 2023-08-11T15:34:19.896 INFO:teuthology.orchestra.run.smithi192.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2023-08-11T15:34:19.955 INFO:teuthology.orchestra.run.smithi192.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 490 kB/s | 196 kB 00:00 2023-08-11T15:34:20.015 INFO:teuthology.orchestra.run.smithi192.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 205 kB/s | 24 kB 00:00 2023-08-11T15:34:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:(23/131): ceph-radosgw-18.0.0-5437.g92684807.el 2.8 MB/s | 14 MB 00:04 2023-08-11T15:34:20.072 INFO:teuthology.orchestra.run.smithi192.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-08-11T15:34:20.079 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-11T15:34:20.090 INFO:teuthology.orchestra.run.smithi032.stdout:(24/131): python3-rgw-18.0.0-5437.g92684807.el8 474 kB/s | 107 kB 00:00 2023-08-11T15:34:20.115 INFO:teuthology.orchestra.run.smithi032.stdout:(25/131): rbd-fuse-18.0.0-5437.g92684807.el8.x8 1.1 MB/s | 94 kB 00:00 2023-08-11T15:34:20.131 INFO:teuthology.orchestra.run.smithi192.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-08-11T15:34:20.207 INFO:teuthology.orchestra.run.smithi032.stdout:(26/131): rbd-nbd-18.0.0-5437.g92684807.el8.x86 1.9 MB/s | 179 kB 00:00 2023-08-11T15:34:20.274 INFO:teuthology.orchestra.run.smithi032.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5437.g 346 kB/s | 23 kB 00:00 2023-08-11T15:34:20.296 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-11T15:34:20.333 INFO:teuthology.orchestra.run.smithi032.stdout:(28/131): rbd-mirror-18.0.0-5437.g92684807.el8. 15 MB/s | 3.5 MB 00:00 2023-08-11T15:34:20.358 INFO:teuthology.orchestra.run.smithi032.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5437.g9268480 1.5 MB/s | 131 kB 00:00 2023-08-11T15:34:20.381 INFO:teuthology.orchestra.run.smithi192.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 733 kB/s | 1.2 MB 00:01 2023-08-11T15:34:20.456 INFO:teuthology.orchestra.run.smithi192.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 665 kB/s | 255 kB 00:00 2023-08-11T15:34:20.467 INFO:teuthology.orchestra.run.smithi032.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5437.g92684 13 MB/s | 1.7 MB 00:00 2023-08-11T15:34:20.490 INFO:teuthology.orchestra.run.smithi192.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 565 kB/s | 61 kB 00:00 2023-08-11T15:34:20.515 INFO:teuthology.orchestra.run.smithi192.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 654 kB/s | 251 kB 00:00 2023-08-11T15:34:20.541 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-11T15:34:20.607 INFO:teuthology.orchestra.run.smithi192.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-08-11T15:34:20.624 INFO:teuthology.orchestra.run.smithi192.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-08-11T15:34:20.660 INFO:teuthology.orchestra.run.smithi192.stdout:(70/131): python3-webtest-2.0.33-1.el8.noarch.r 424 kB/s | 85 kB 00:00 2023-08-11T15:34:20.709 INFO:teuthology.orchestra.run.smithi032.stdout:(31/131): ceph-common-18.0.0-5437.g92684807.el8 2.2 MB/s | 22 MB 00:10 2023-08-11T15:34:20.726 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-11T15:34:20.736 INFO:teuthology.orchestra.run.smithi032.stdout:(32/131): ceph-mgr-modules-core-18.0.0-5437.g92 904 kB/s | 241 kB 00:00 2023-08-11T15:34:20.760 INFO:teuthology.orchestra.run.smithi032.stdout:(33/131): ceph-mgr-rook-18.0.0-5437.g92684807.e 949 kB/s | 47 kB 00:00 2023-08-11T15:34:20.761 INFO:teuthology.orchestra.run.smithi192.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-08-11T15:34:20.785 INFO:teuthology.orchestra.run.smithi032.stdout:(34/131): ceph-prometheus-alerts-18.0.0-5437.g9 291 kB/s | 15 kB 00:00 2023-08-11T15:34:20.792 INFO:teuthology.orchestra.run.smithi192.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-08-11T15:34:20.811 INFO:teuthology.orchestra.run.smithi032.stdout:(35/131): ceph-volume-18.0.0-5437.g92684807.el8 5.1 MB/s | 261 kB 00:00 2023-08-11T15:34:20.843 INFO:teuthology.orchestra.run.smithi192.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-08-11T15:34:20.852 INFO:teuthology.orchestra.run.smithi032.stdout:(36/131): cephadm-18.0.0-5437.g92684807.el8.noa 2.5 MB/s | 170 kB 00:00 2023-08-11T15:34:20.877 INFO:teuthology.orchestra.run.smithi032.stdout:(37/131): cephfs-top-18.0.0-5437.g92684807.el8. 269 kB/s | 18 kB 00:00 2023-08-11T15:34:20.917 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-11T15:34:20.917 INFO:teuthology.orchestra.run.smithi192.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-08-11T15:34:20.959 INFO:teuthology.orchestra.run.smithi192.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 932 kB/s | 39 kB 00:00 2023-08-11T15:34:21.102 INFO:teuthology.orchestra.run.smithi192.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 29 MB/s | 14 MB 00:00 2023-08-11T15:34:21.135 INFO:teuthology.orchestra.run.smithi192.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2023-08-11T15:34:21.169 INFO:teuthology.orchestra.run.smithi192.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-08-11T15:34:21.194 INFO:teuthology.orchestra.run.smithi192.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-08-11T15:34:21.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-11T15:34:21.219 INFO:teuthology.orchestra.run.smithi192.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-08-11T15:34:21.261 INFO:teuthology.orchestra.run.smithi192.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-08-11T15:34:21.303 INFO:teuthology.orchestra.run.smithi192.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 637 kB/s | 27 kB 00:00 2023-08-11T15:34:21.336 INFO:teuthology.orchestra.run.smithi032.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 128 kB/s | 59 kB 00:00 2023-08-11T15:34:21.368 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 26/134 2023-08-11T15:34:21.387 INFO:teuthology.orchestra.run.smithi192.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-08-11T15:34:21.412 INFO:teuthology.orchestra.run.smithi192.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-08-11T15:34:21.437 INFO:teuthology.orchestra.run.smithi192.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 877 kB/s | 44 kB 00:00 2023-08-11T15:34:21.489 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 27/134 2023-08-11T15:34:21.503 INFO:teuthology.orchestra.run.smithi032.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 471 kB/s | 306 kB 00:00 2023-08-11T15:34:21.528 INFO:teuthology.orchestra.run.smithi032.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 392 kB/s | 75 kB 00:00 2023-08-11T15:34:21.637 INFO:teuthology.orchestra.run.smithi032.stdout:(41/131): ceph-mgr-diskprediction-local-18.0.0- 5.8 MB/s | 7.4 MB 00:01 2023-08-11T15:34:21.662 INFO:teuthology.orchestra.run.smithi032.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 326 kB/s | 44 kB 00:00 2023-08-11T15:34:21.721 INFO:teuthology.orchestra.run.smithi192.stdout:(86/131): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-08-11T15:34:21.771 INFO:teuthology.orchestra.run.smithi032.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2023-08-11T15:34:21.796 INFO:teuthology.orchestra.run.smithi192.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 1.2 MB/s | 457 kB 00:00 2023-08-11T15:34:21.822 INFO:teuthology.orchestra.run.smithi192.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-08-11T15:34:21.847 INFO:teuthology.orchestra.run.smithi192.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-08-11T15:34:21.872 INFO:teuthology.orchestra.run.smithi192.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-08-11T15:34:21.879 INFO:teuthology.orchestra.run.smithi032.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-08-11T15:34:21.897 INFO:teuthology.orchestra.run.smithi192.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-08-11T15:34:21.905 INFO:teuthology.orchestra.run.smithi032.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 122 kB/s | 16 kB 00:00 2023-08-11T15:34:21.939 INFO:teuthology.orchestra.run.smithi192.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2023-08-11T15:34:21.973 INFO:teuthology.orchestra.run.smithi192.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-08-11T15:34:22.023 INFO:teuthology.orchestra.run.smithi192.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-08-11T15:34:22.080 INFO:teuthology.orchestra.run.smithi032.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 862 kB/s | 173 kB 00:00 2023-08-11T15:34:22.124 INFO:teuthology.orchestra.run.smithi192.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 26 MB/s | 4.9 MB 00:00 2023-08-11T15:34:22.166 INFO:teuthology.orchestra.run.smithi192.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-08-11T15:34:22.189 INFO:teuthology.orchestra.run.smithi032.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 752 kB/s | 81 kB 00:00 2023-08-11T15:34:22.207 INFO:teuthology.orchestra.run.smithi192.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-08-11T15:34:22.214 INFO:teuthology.orchestra.run.smithi032.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-08-11T15:34:22.239 INFO:teuthology.orchestra.run.smithi032.stdout:(49/131): python3-beautifulsoup4-4.6.3-2.el8.1. 308 kB/s | 185 kB 00:00 2023-08-11T15:34:22.291 INFO:teuthology.orchestra.run.smithi192.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-08-11T15:34:22.298 INFO:teuthology.orchestra.run.smithi032.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-08-11T15:34:22.323 INFO:teuthology.orchestra.run.smithi032.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-08-11T15:34:22.358 INFO:teuthology.orchestra.run.smithi192.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-08-11T15:34:22.407 INFO:teuthology.orchestra.run.smithi032.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2023-08-11T15:34:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 28/134 2023-08-11T15:34:22.432 INFO:teuthology.orchestra.run.smithi032.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 540 kB/s | 59 kB 00:00 2023-08-11T15:34:22.455 INFO:teuthology.orchestra.run.smithi192.stdout:(100/131): postgresql-server-10.21-2.module+el8 21 MB/s | 5.1 MB 00:00 2023-08-11T15:34:22.492 INFO:teuthology.orchestra.run.smithi192.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-08-11T15:34:22.517 INFO:teuthology.orchestra.run.smithi192.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2023-08-11T15:34:22.543 INFO:teuthology.orchestra.run.smithi192.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-08-11T15:34:22.550 INFO:teuthology.orchestra.run.smithi032.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 422 kB/s | 60 kB 00:00 2023-08-11T15:34:22.568 INFO:teuthology.orchestra.run.smithi192.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-08-11T15:34:22.594 INFO:teuthology.orchestra.run.smithi192.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-08-11T15:34:22.619 INFO:teuthology.orchestra.run.smithi192.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 954 kB/s | 48 kB 00:00 2023-08-11T15:34:22.635 INFO:teuthology.orchestra.run.smithi192.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-08-11T15:34:22.661 INFO:teuthology.orchestra.run.smithi192.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-08-11T15:34:22.666 INFO:teuthology.orchestra.run.smithi032.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 134 kB/s | 16 kB 00:00 2023-08-11T15:34:22.686 INFO:teuthology.orchestra.run.smithi192.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-08-11T15:34:22.692 INFO:teuthology.orchestra.run.smithi032.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-08-11T15:34:22.712 INFO:teuthology.orchestra.run.smithi192.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-08-11T15:34:22.732 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-11T15:34:22.745 INFO:teuthology.orchestra.run.smithi192.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-08-11T15:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 372 kB/s | 19 kB 00:00 2023-08-11T15:34:22.775 INFO:teuthology.orchestra.run.smithi032.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-08-11T15:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-08-11T15:34:22.829 INFO:teuthology.orchestra.run.smithi192.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.1 MB/s | 559 kB 00:00 2023-08-11T15:34:22.846 INFO:teuthology.orchestra.run.smithi192.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 355 kB/s | 18 kB 00:00 2023-08-11T15:34:22.884 INFO:teuthology.orchestra.run.smithi032.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 593 kB/s | 64 kB 00:00 2023-08-11T15:34:22.897 INFO:teuthology.orchestra.run.smithi192.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-08-11T15:34:22.909 INFO:teuthology.orchestra.run.smithi032.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 905 kB/s | 196 kB 00:00 2023-08-11T15:34:22.922 INFO:teuthology.orchestra.run.smithi192.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-08-11T15:34:22.951 INFO:teuthology.orchestra.run.smithi032.stdout:(60/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-08-11T15:34:22.972 INFO:teuthology.orchestra.run.smithi192.stdout:(118/131): thrift-0.13.0-2.el8.x86_64.rpm 723 kB/s | 1.7 MB 00:02 2023-08-11T15:34:22.989 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-11T15:34:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-08-11T15:34:23.031 INFO:teuthology.orchestra.run.smithi192.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 17 MB/s | 2.3 MB 00:00 2023-08-11T15:34:23.043 INFO:teuthology.orchestra.run.smithi032.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 179 kB/s | 24 kB 00:00 2023-08-11T15:34:23.056 INFO:teuthology.orchestra.run.smithi192.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 539 kB/s | 72 kB 00:00 2023-08-11T15:34:23.069 INFO:teuthology.orchestra.run.smithi032.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 249 kB/s | 29 kB 00:00 2023-08-11T15:34:23.098 INFO:teuthology.orchestra.run.smithi192.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 896 kB/s | 60 kB 00:00 2023-08-11T15:34:23.102 INFO:teuthology.orchestra.run.smithi032.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 252 kB/s | 27 kB 00:00 2023-08-11T15:34:23.132 INFO:teuthology.orchestra.run.smithi192.stdout:(122/131): fuse-2.9.7-15.el8.x86_64.rpm 519 kB/s | 83 kB 00:00 2023-08-11T15:34:23.174 INFO:teuthology.orchestra.run.smithi192.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 837 kB/s | 98 kB 00:00 2023-08-11T15:34:23.199 INFO:teuthology.orchestra.run.smithi192.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 573 kB/s | 58 kB 00:00 2023-08-11T15:34:23.211 INFO:teuthology.orchestra.run.smithi032.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 565 kB/s | 61 kB 00:00 2023-08-11T15:34:23.233 INFO:teuthology.orchestra.run.smithi192.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 384 kB/s | 39 kB 00:00 2023-08-11T15:34:23.237 INFO:teuthology.orchestra.run.smithi032.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.3 MB/s | 255 kB 00:00 2023-08-11T15:34:23.241 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-11T15:34:23.270 INFO:teuthology.orchestra.run.smithi032.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-08-11T15:34:23.317 INFO:teuthology.orchestra.run.smithi192.stdout:(126/131): NetworkManager-initscripts-updown-1. 1.6 MB/s | 138 kB 00:00 2023-08-11T15:34:23.320 INFO:teuthology.orchestra.run.smithi032.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 785 kB/s | 85 kB 00:00 2023-08-11T15:34:23.337 INFO:teuthology.orchestra.run.smithi032.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-08-11T15:34:23.342 INFO:teuthology.orchestra.run.smithi192.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.2 MB/s | 171 kB 00:00 2023-08-11T15:34:23.363 INFO:teuthology.orchestra.run.smithi032.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-08-11T15:34:23.375 INFO:teuthology.orchestra.run.smithi192.stdout:(128/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.1 MB/s | 644 kB 00:00 2023-08-11T15:34:23.434 INFO:teuthology.orchestra.run.smithi192.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-08-11T15:34:23.438 INFO:teuthology.orchestra.run.smithi032.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 467 kB/s | 47 kB 00:00 2023-08-11T15:34:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T15:34:23.463 INFO:teuthology.orchestra.run.smithi032.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-08-11T15:34:23.605 INFO:teuthology.orchestra.run.smithi032.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2023-08-11T15:34:23.630 INFO:teuthology.orchestra.run.smithi032.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-08-11T15:34:23.697 INFO:teuthology.orchestra.run.smithi032.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 522 kB/s | 48 kB 00:00 2023-08-11T15:34:23.734 INFO:teuthology.orchestra.run.smithi192.stdout:(130/131): librados2-18.0.0-5437.g92684807.el8. 8.8 MB/s | 3.7 MB 00:00 2023-08-11T15:34:23.739 INFO:teuthology.orchestra.run.smithi032.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 803 kB/s | 87 kB 00:00 2023-08-11T15:34:23.765 INFO:teuthology.orchestra.run.smithi032.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 639 kB/s | 43 kB 00:00 2023-08-11T15:34:23.790 INFO:teuthology.orchestra.run.smithi032.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-08-11T15:34:23.793 INFO:teuthology.orchestra.run.smithi192.stdout:(131/131): librbd1-18.0.0-5437.g92684807.el8.x8 8.1 MB/s | 3.6 MB 00:00 2023-08-11T15:34:23.797 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:34:23.797 INFO:teuthology.orchestra.run.smithi192.stdout:Total 14 MB/s | 206 MB 00:14 2023-08-11T15:34:23.882 INFO:teuthology.orchestra.run.smithi032.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-08-11T15:34:23.924 INFO:teuthology.orchestra.run.smithi032.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-08-11T15:34:24.016 INFO:teuthology.orchestra.run.smithi032.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 140 kB/s | 19 kB 00:00 2023-08-11T15:34:24.058 INFO:teuthology.orchestra.run.smithi032.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2023-08-11T15:34:24.083 INFO:teuthology.orchestra.run.smithi032.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 973 kB/s | 155 kB 00:00 2023-08-11T15:34:24.209 INFO:teuthology.orchestra.run.smithi032.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 2.8 MB/s | 559 kB 00:00 2023-08-11T15:34:24.267 INFO:teuthology.orchestra.run.smithi032.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.4 MB/s | 605 kB 00:00 2023-08-11T15:34:24.309 INFO:teuthology.orchestra.run.smithi032.stdout:(86/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 71 kB/s | 18 kB 00:00 2023-08-11T15:34:24.368 INFO:teuthology.orchestra.run.smithi032.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2023-08-11T15:34:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 480 kB/s | 72 kB 00:00 2023-08-11T15:34:24.520 INFO:teuthology.orchestra.run.smithi032.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 544 kB/s | 83 kB 00:00 2023-08-11T15:34:24.594 INFO:teuthology.orchestra.run.smithi032.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 7.1 MB/s | 2.3 MB 00:00 2023-08-11T15:34:24.619 INFO:teuthology.orchestra.run.smithi032.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 377 kB/s | 60 kB 00:00 2023-08-11T15:34:24.636 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-08-11T15:34:24.645 INFO:teuthology.orchestra.run.smithi032.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 794 kB/s | 98 kB 00:00 2023-08-11T15:34:24.755 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-08-11T15:34:24.756 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-08-11T15:34:24.804 INFO:teuthology.orchestra.run.smithi032.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 276 kB/s | 58 kB 00:00 2023-08-11T15:34:24.828 INFO:teuthology.orchestra.run.smithi032.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 185 kB/s | 39 kB 00:00 2023-08-11T15:34:24.862 INFO:teuthology.orchestra.run.smithi032.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-08-11T15:34:24.904 INFO:teuthology.orchestra.run.smithi032.stdout:(96/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 2.4 MB/s | 644 kB 00:00 2023-08-11T15:34:25.021 INFO:teuthology.orchestra.run.smithi032.stdout:(97/131): NetworkManager-initscripts-updown-1.3 715 kB/s | 138 kB 00:00 2023-08-11T15:34:25.063 INFO:teuthology.orchestra.run.smithi032.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 694 kB/s | 110 kB 00:00 2023-08-11T15:34:25.147 INFO:teuthology.orchestra.run.smithi032.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 433 kB/s | 54 kB 00:00 2023-08-11T15:34:25.197 INFO:teuthology.orchestra.run.smithi032.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2023-08-11T15:34:25.223 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T15:34:25.231 INFO:teuthology.orchestra.run.smithi032.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2023-08-11T15:34:25.314 INFO:teuthology.orchestra.run.smithi032.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 332 kB/s | 39 kB 00:00 2023-08-11T15:34:25.531 INFO:teuthology.orchestra.run.smithi032.stdout:(103/131): python3-scipy-1.0.0-21.module+el8.5. 20 MB/s | 14 MB 00:00 2023-08-11T15:34:25.545 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-11T15:34:25.565 INFO:teuthology.orchestra.run.smithi032.stdout:(104/131): bison-3.0.4-10.el8.x86_64.rpm 2.0 MB/s | 688 kB 00:00 2023-08-11T15:34:25.598 INFO:teuthology.orchestra.run.smithi032.stdout:(105/131): python3-jinja2-2.10.1-3.el8.noarch.r 1.9 MB/s | 538 kB 00:00 2023-08-11T15:34:25.682 INFO:teuthology.orchestra.run.smithi032.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2023-08-11T15:34:25.707 INFO:teuthology.orchestra.run.smithi032.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-08-11T15:34:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-11T15:34:25.850 INFO:teuthology.orchestra.run.smithi032.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 187 kB/s | 27 kB 00:00 2023-08-11T15:34:25.879 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-08-11T15:34:25.883 INFO:teuthology.orchestra.run.smithi032.stdout:(109/131): automake-1.16.1-7.el8.noarch.rpm 2.4 MB/s | 713 kB 00:00 2023-08-11T15:34:25.884 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-08-11T15:34:25.987 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-11T15:34:26.017 INFO:teuthology.orchestra.run.smithi032.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-08-11T15:34:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 263 kB/s | 44 kB 00:00 2023-08-11T15:34:26.084 INFO:teuthology.orchestra.run.smithi032.stdout:(112/131): flex-2.6.1-9.el8.x86_64.rpm 1.3 MB/s | 320 kB 00:00 2023-08-11T15:34:26.201 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 36/134 2023-08-11T15:34:26.209 INFO:teuthology.orchestra.run.smithi032.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2023-08-11T15:34:26.235 INFO:teuthology.orchestra.run.smithi032.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2023-08-11T15:34:26.268 INFO:teuthology.orchestra.run.smithi032.stdout:(115/131): python3-werkzeug-0.12.2-4.el8.noarch 2.1 MB/s | 457 kB 00:00 2023-08-11T15:34:26.327 INFO:teuthology.orchestra.run.smithi032.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 547 kB/s | 64 kB 00:00 2023-08-11T15:34:26.352 INFO:teuthology.orchestra.run.smithi032.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 239 kB/s | 20 kB 00:00 2023-08-11T15:34:26.373 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 37/134 2023-08-11T15:34:26.386 INFO:teuthology.orchestra.run.smithi032.stdout:(118/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 839 kB/s | 126 kB 00:00 2023-08-11T15:34:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-08-11T15:34:26.510 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-11T15:34:26.512 INFO:teuthology.orchestra.run.smithi032.stdout:(120/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 563 kB/s | 103 kB 00:00 2023-08-11T15:34:26.554 INFO:teuthology.orchestra.run.smithi032.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2023-08-11T15:34:26.695 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-11T15:34:26.696 INFO:teuthology.orchestra.run.smithi032.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 16 MB/s | 4.9 MB 00:00 2023-08-11T15:34:26.787 INFO:teuthology.orchestra.run.smithi032.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 865 kB/s | 202 kB 00:00 2023-08-11T15:34:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-08-11T15:34:26.913 INFO:teuthology.orchestra.run.smithi032.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.6 MB/s | 354 kB 00:00 2023-08-11T15:34:26.930 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-11T15:34:27.039 INFO:teuthology.orchestra.run.smithi032.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.0 MB/s | 808 kB 00:00 2023-08-11T15:34:27.073 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-11T15:34:27.139 INFO:teuthology.orchestra.run.smithi032.stdout:(127/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-08-11T15:34:27.197 INFO:teuthology.orchestra.run.smithi032.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 5.3 MB/s | 1.5 MB 00:00 2023-08-11T15:34:27.293 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-11T15:34:27.331 INFO:teuthology.orchestra.run.smithi032.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.8 MB/s | 798 kB 00:00 2023-08-11T15:34:27.423 INFO:teuthology.orchestra.run.smithi032.stdout:(130/131): librados2-18.0.0-5437.g92684807.el8. 9.6 MB/s | 3.7 MB 00:00 2023-08-11T15:34:27.509 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-11T15:34:27.756 INFO:teuthology.orchestra.run.smithi032.stdout:(131/131): librbd1-18.0.0-5437.g92684807.el8.x8 5.9 MB/s | 3.6 MB 00:00 2023-08-11T15:34:27.760 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:34:27.761 INFO:teuthology.orchestra.run.smithi032.stdout:Total 12 MB/s | 206 MB 00:17 2023-08-11T15:34:27.894 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-11T15:34:28.155 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-11T15:34:28.345 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-11T15:34:28.590 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-11T15:34:28.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-11T15:34:29.117 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-11T15:34:29.327 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-11T15:34:29.455 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-08-11T15:34:29.516 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-11T15:34:29.602 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-08-11T15:34:29.602 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-08-11T15:34:29.782 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-11T15:34:29.957 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-11T15:34:30.114 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-11T15:34:30.457 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-11T15:34:30.709 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-11T15:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-11T15:34:31.062 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-08-11T15:34:31.192 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-11T15:34:31.220 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-08-11T15:34:31.225 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-08-11T15:34:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-11T15:34:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-11T15:34:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-11T15:34:31.561 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-11T15:34:31.637 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-11T15:34:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T15:34:31.837 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T15:34:31.847 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T15:34:31.928 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T15:34:32.125 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-11T15:34:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T15:34:32.329 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-11T15:34:32.518 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T15:34:32.601 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T15:34:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T15:34:32.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-11T15:34:32.921 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:34:32.978 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-11T15:34:33.168 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-11T15:34:33.330 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:34:33.365 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-11T15:34:33.529 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:34:33.610 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-11T15:34:33.654 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:34:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-11T15:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:34:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:34:34.031 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-11T15:34:34.188 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-11T15:34:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T15:34:34.412 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-11T15:34:34.449 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-11T15:34:34.614 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-11T15:34:34.681 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-11T15:34:34.789 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-11T15:34:34.979 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-11T15:34:35.087 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-11T15:34:35.124 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-11T15:34:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-11T15:34:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-11T15:34:35.470 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T15:34:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T15:34:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-11T15:34:35.728 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T15:34:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-11T15:34:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T15:34:35.978 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T15:34:36.074 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-11T15:34:36.113 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T15:34:36.292 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-11T15:34:36.303 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T15:34:36.466 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-11T15:34:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-08-11T15:34:36.702 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-11T15:34:36.727 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-11T15:34:36.898 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-11T15:34:36.917 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T15:34:37.069 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-11T15:34:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T15:34:37.217 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-11T15:34:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-11T15:34:37.738 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T15:34:37.825 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-11T15:34:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-11T15:34:38.096 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-11T15:34:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el8.x86_64 5/134 2023-08-11T15:34:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T15:34:38.384 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-11T15:34:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el8.x86_64 5/134 2023-08-11T15:34:38.693 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-11T15:34:38.786 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:34:38.830 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T15:34:38.937 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:34:39.002 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-11T15:34:39.126 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-11T15:34:39.127 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:34:39.192 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-11T15:34:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:34:39.355 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-11T15:34:39.450 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:34:39.590 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-11T15:34:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-08-11T15:34:39.792 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-11T15:34:39.900 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-11T15:34:39.978 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-11T15:34:40.050 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-11T15:34:40.159 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T15:34:40.159 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-11T15:34:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T15:34:40.306 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-11T15:34:40.417 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 93/134 2023-08-11T15:34:40.492 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 26/134 2023-08-11T15:34:40.600 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-11T15:34:40.631 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 27/134 2023-08-11T15:34:40.882 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-11T15:34:41.019 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T15:34:41.136 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T15:34:41.268 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T15:34:41.448 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T15:34:41.551 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 28/134 2023-08-11T15:34:41.660 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-08-11T15:34:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-08-11T15:34:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-11T15:34:42.047 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-08-11T15:34:42.162 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-11T15:34:42.176 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-11T15:34:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-11T15:34:42.440 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-11T15:34:42.543 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-11T15:34:42.669 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T15:34:43.296 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-11T15:34:43.468 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-11T15:34:44.119 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:34:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-11T15:34:44.325 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:34:44.393 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-08-11T15:34:44.428 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T15:34:44.744 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-11T15:34:44.902 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:34:44.947 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-11T15:34:45.129 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-11T15:34:45.216 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T15:34:45.296 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T15:34:45.297 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-08-11T15:34:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 36/134 2023-08-11T15:34:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-08-11T15:34:45.572 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 37/134 2023-08-11T15:34:45.598 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T15:34:45.743 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-08-11T15:34:45.751 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-11T15:34:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-11T15:34:45.994 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-11T15:34:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-11T15:34:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 26/134 2023-08-11T15:34:46.297 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-11T15:34:46.316 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 27/134 2023-08-11T15:34:46.562 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 28/134 2023-08-11T15:34:46.584 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-11T15:34:46.768 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-11T15:34:46.783 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-11T15:34:47.150 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-11T15:34:47.430 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-11T15:34:47.653 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-11T15:34:47.881 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-11T15:34:48.123 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-11T15:34:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-11T15:34:48.684 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-11T15:34:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-11T15:34:49.021 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-11T15:34:49.114 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-11T15:34:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-11T15:34:49.496 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-11T15:34:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-11T15:34:50.054 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-11T15:34:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-11T15:34:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-08-11T15:34:50.608 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-11T15:34:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-08-11T15:34:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-11T15:34:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-11T15:34:50.982 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-08-11T15:34:51.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T15:34:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T15:34:51.257 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-08-11T15:34:51.438 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-11T15:34:51.460 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-11T15:34:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-11T15:34:51.684 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-11T15:34:51.809 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T15:34:51.868 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 36/134 2023-08-11T15:34:51.890 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T15:34:52.051 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 37/134 2023-08-11T15:34:52.149 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-11T15:34:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-08-11T15:34:52.345 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-11T15:34:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-08-11T15:34:52.510 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-11T15:34:52.534 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-08-11T15:34:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-08-11T15:34:52.682 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-11T15:34:52.931 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-08-11T15:34:52.958 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-11T15:34:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-08-11T15:34:53.191 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-11T15:34:53.292 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-11T15:34:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-11T15:34:53.397 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-11T15:34:53.521 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-11T15:34:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-08-11T15:34:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-11T15:34:53.806 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-08-11T15:34:53.952 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-08-11T15:34:53.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-11T15:34:54.155 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-11T15:34:54.164 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-08-11T15:34:54.320 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-08-11T15:34:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-11T15:34:54.492 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-08-11T15:34:54.532 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-11T15:34:54.745 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-08-11T15:34:54.797 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-11T15:34:54.980 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-08-11T15:34:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-11T15:34:55.215 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-11T15:34:55.449 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-08-11T15:34:55.465 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-11T15:34:55.666 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-11T15:34:55.849 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-11T15:34:55.970 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-08-11T15:34:56.108 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-11T15:34:56.263 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-08-11T15:34:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-08-11T15:34:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-08-11T15:34:57.078 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-08-11T15:34:57.233 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-11T15:34:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-08-11T15:34:57.490 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-08-11T15:34:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-11T15:34:57.723 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-08-11T15:34:57.816 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-11T15:34:57.885 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-08-11T15:34:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-08-11T15:34:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-11T15:34:58.419 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-08-11T15:34:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-11T15:34:58.647 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-11T15:34:58.707 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-08-11T15:34:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-08-11T15:34:59.229 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-08-11T15:34:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-11T15:34:59.455 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-11T15:34:59.559 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-11T15:34:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T15:34:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T15:34:59.859 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-11T15:34:59.884 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 93/134 2023-08-11T15:35:00.059 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-11T15:35:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-11T15:35:00.389 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-11T15:35:00.627 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-11T15:35:00.828 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-11T15:35:01.001 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-11T15:35:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-11T15:35:01.375 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-11T15:35:01.590 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-11T15:35:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-11T15:35:02.141 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-11T15:35:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-11T15:35:02.660 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-11T15:35:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-11T15:35:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-11T15:35:03.605 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:35:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:35:04.435 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:35:04.480 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T15:35:04.480 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2023-08-11T15:35:04.480 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /run 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-08-11T15:35:04.481 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:35:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-11T15:35:04.830 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T15:35:04.857 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-11T15:35:05.141 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-11T15:35:05.326 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T15:35:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T15:35:05.515 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-11T15:35:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T15:35:05.841 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-11T15:35:06.056 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-11T15:35:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T15:35:06.580 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T15:35:06.798 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T15:35:06.953 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-11T15:35:06.996 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T15:35:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T15:35:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T15:35:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 93/134 2023-08-11T15:35:07.450 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T15:35:07.604 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T15:35:07.621 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T15:35:09.286 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T15:35:09.490 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T15:35:09.610 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 102/134 2023-08-11T15:35:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 103/134 2023-08-11T15:35:10.028 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T15:35:10.244 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T15:35:10.546 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T15:35:11.505 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T15:35:11.687 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T15:35:11.777 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T15:35:12.132 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 107/134 2023-08-11T15:35:12.348 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T15:35:12.382 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T15:35:12.467 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:35:12.536 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T15:35:12.548 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T15:35:12.682 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:35:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T15:35:13.693 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T15:35:13.834 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T15:35:13.879 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el8.x86_64 110/134 2023-08-11T15:35:13.998 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T15:35:14.071 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T15:35:14.105 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T15:35:14.253 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T15:35:14.298 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T15:35:14.487 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T15:35:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T15:35:15.029 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T15:35:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T15:35:19.870 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 115/134 2023-08-11T15:35:20.035 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T15:35:20.239 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T15:35:21.111 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-11T15:35:21.475 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-11T15:35:21.638 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 119/134 2023-08-11T15:35:21.858 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T15:35:22.015 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 121/134 2023-08-11T15:35:22.170 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T15:35:24.353 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:35:24.430 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T15:35:24.431 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2023-08-11T15:35:24.431 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2023-08-11T15:35:24.431 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2023-08-11T15:35:24.431 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /run 2023-08-11T15:35:24.431 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2023-08-11T15:35:24.431 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2023-08-11T15:35:24.432 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2023-08-11T15:35:24.432 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2023-08-11T15:35:24.432 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2023-08-11T15:35:24.432 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:35:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:35:24.610 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:35:24.810 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 124/134 2023-08-11T15:35:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T15:35:25.118 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 125/134 2023-08-11T15:35:25.428 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 126/134 2023-08-11T15:35:25.695 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-11T15:35:25.826 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T15:35:25.834 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T15:35:26.048 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-11T15:35:26.243 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T15:35:26.328 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T15:35:26.421 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T15:35:26.560 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T15:35:26.572 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T15:35:26.572 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:35:26.597 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T15:35:26.771 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:35:26.771 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T15:35:26.816 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T15:35:26.999 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T15:35:27.013 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T15:35:27.242 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:35:27.467 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T15:35:27.637 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T15:35:27.654 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T15:35:27.757 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:35:27.757 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el8.x86_64 1/134 2023-08-11T15:35:27.758 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2/134 2023-08-11T15:35:27.758 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 3/134 2023-08-11T15:35:27.758 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 4/134 2023-08-11T15:35:27.758 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/134 2023-08-11T15:35:27.758 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:35:27.758 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:35:27.758 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:35:27.759 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T15:35:27.759 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 10/134 2023-08-11T15:35:27.759 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 11/134 2023-08-11T15:35:27.759 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 12/134 2023-08-11T15:35:27.759 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 13/134 2023-08-11T15:35:27.759 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 14/134 2023-08-11T15:35:27.759 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 15/134 2023-08-11T15:35:27.760 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 16/134 2023-08-11T15:35:27.760 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 17/134 2023-08-11T15:35:27.760 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 18/134 2023-08-11T15:35:27.760 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 19/134 2023-08-11T15:35:27.761 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/134 2023-08-11T15:35:27.761 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 21/134 2023-08-11T15:35:27.761 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 22/134 2023-08-11T15:35:27.761 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 23/134 2023-08-11T15:35:27.762 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 24/134 2023-08-11T15:35:27.762 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 25/134 2023-08-11T15:35:27.762 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 26/134 2023-08-11T15:35:27.763 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 27/134 2023-08-11T15:35:27.763 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 28/134 2023-08-11T15:35:27.763 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/134 2023-08-11T15:35:27.763 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 30/134 2023-08-11T15:35:27.764 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 31/134 2023-08-11T15:35:27.764 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/134 2023-08-11T15:35:27.764 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/134 2023-08-11T15:35:27.764 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 34/134 2023-08-11T15:35:27.765 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/134 2023-08-11T15:35:27.765 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 36/134 2023-08-11T15:35:27.765 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el8.noarch 37/134 2023-08-11T15:35:27.766 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 38/134 2023-08-11T15:35:27.766 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-11T15:35:27.766 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-11T15:35:27.766 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-11T15:35:27.766 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-11T15:35:27.766 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-11T15:35:27.766 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-11T15:35:27.767 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-11T15:35:27.767 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-11T15:35:27.767 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-11T15:35:27.767 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-11T15:35:27.767 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-11T15:35:27.767 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-11T15:35:27.767 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-11T15:35:27.768 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-11T15:35:27.768 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-11T15:35:27.768 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-11T15:35:27.768 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-11T15:35:27.768 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-11T15:35:27.768 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-11T15:35:27.768 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-11T15:35:27.769 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-11T15:35:27.769 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-11T15:35:27.769 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-11T15:35:27.769 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-11T15:35:27.769 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-11T15:35:27.769 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-11T15:35:27.770 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-11T15:35:27.771 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-11T15:35:27.771 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-11T15:35:27.771 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-11T15:35:27.771 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-11T15:35:27.772 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-11T15:35:27.772 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-11T15:35:27.772 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-11T15:35:27.772 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-11T15:35:27.772 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-11T15:35:27.772 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-11T15:35:27.772 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-11T15:35:27.773 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-11T15:35:27.773 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-11T15:35:27.773 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-11T15:35:27.773 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-11T15:35:27.773 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-11T15:35:27.773 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-11T15:35:27.773 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-11T15:35:27.774 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-11T15:35:27.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-11T15:35:27.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-11T15:35:27.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-11T15:35:27.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-11T15:35:27.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-11T15:35:27.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-11T15:35:27.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-11T15:35:27.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-11T15:35:27.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-11T15:35:27.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-11T15:35:27.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-11T15:35:27.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-11T15:35:27.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-11T15:35:27.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-11T15:35:27.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-11T15:35:27.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-11T15:35:27.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-11T15:35:27.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-11T15:35:27.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-11T15:35:27.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-11T15:35:27.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-11T15:35:27.778 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-11T15:35:27.779 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T15:35:27.779 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el8.x86_64 129/134 2023-08-11T15:35:27.779 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-11T15:35:27.779 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 131/134 2023-08-11T15:35:27.779 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:35:27.779 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-11T15:35:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T15:35:29.566 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T15:35:29.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 102/134 2023-08-11T15:35:29.870 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 103/134 2023-08-11T15:35:30.070 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T15:35:30.252 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:35:30.253 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:48 ago on Fri 11 Aug 2023 03:33:42 PM UTC. 2023-08-11T15:35:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T15:35:30.571 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T15:35:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T15:35:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T15:35:31.844 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T15:35:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 107/134 2023-08-11T15:35:32.440 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T15:35:32.476 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T15:35:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T15:35:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T15:35:32.990 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-08-11T15:35:33.206 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:35:33.206 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2023-08-11T15:35:33.206 INFO:teuthology.orchestra.run.smithi134.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-11T15:35:33.207 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.207 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.207 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-08-11T15:35:33.207 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-11T15:35:33.207 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.1-7.el8.noarch 2023-08-11T15:35:33.207 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.0.4-10.el8.x86_64 2023-08-11T15:35:33.207 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.208 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.208 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.208 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.208 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.208 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.208 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.208 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.209 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.209 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.209 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.209 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.209 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.209 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.209 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:33.211 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-11T15:35:33.211 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.1-9.el8.x86_64 2023-08-11T15:35:33.211 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-11T15:35:33.211 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-11T15:35:33.211 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-3.el8.x86_64 2023-08-11T15:35:33.211 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-11T15:35:33.211 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-11T15:35:33.212 INFO:teuthology.orchestra.run.smithi134.stdout: libpq-13.5-1.el8.x86_64 2023-08-11T15:35:33.213 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-11T15:35:33.213 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-11T15:35:33.213 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.213 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.213 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-11T15:35:33.213 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.213 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-11T15:35:33.214 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-11T15:35:33.215 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-11T15:35:33.215 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-11T15:35:33.215 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-11T15:35:33.215 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-11T15:35:33.215 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-11T15:35:33.215 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:35:33.215 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:35:33.216 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:35:33.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-11T15:35:33.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-11T15:35:33.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-11T15:35:33.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-11T15:35:33.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-11T15:35:33.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-11T15:35:33.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-11T15:35:33.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-11T15:35:33.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-11T15:35:33.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-11T15:35:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-11T15:35:33.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-11T15:35:33.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-11T15:35:33.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-11T15:35:33.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-11T15:35:33.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-11T15:35:33.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-11T15:35:33.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-11T15:35:33.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-11T15:35:33.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-11T15:35:33.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-2.6-1.el8.noarch 2023-08-11T15:35:33.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-11T15:35:33.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-11T15:35:33.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-11T15:35:33.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-11T15:35:33.221 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-11T15:35:33.221 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-11T15:35:33.221 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.221 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.221 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-11T15:35:33.221 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-11T15:35:33.221 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.222 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-11T15:35:33.222 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-11T15:35:33.222 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-11T15:35:33.222 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-11T15:35:33.222 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-11T15:35:33.222 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-11T15:35:33.222 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-11T15:35:33.223 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-11T15:35:33.223 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-11T15:35:33.223 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-11T15:35:33.223 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-11T15:35:33.223 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-11T15:35:33.223 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-11T15:35:33.223 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.224 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.224 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:33.224 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-11T15:35:33.224 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-11T15:35:33.224 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-11T15:35:33.224 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-11T15:35:33.224 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-11T15:35:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-11T15:35:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-08-11T15:35:33.225 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-08-11T15:35:33.393 DEBUG:teuthology.parallel:result is None 2023-08-11T15:35:33.967 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el8.x86_64 110/134 2023-08-11T15:35:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T15:35:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T15:35:34.386 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T15:35:34.435 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T15:35:34.595 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T15:35:34.947 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T15:35:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T15:35:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2023-08-11T15:35:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2023-08-11T15:35:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-08-11T15:35:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /run 2023-08-11T15:35:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2023-08-11T15:35:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2023-08-11T15:35:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2023-08-11T15:35:35.022 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2023-08-11T15:35:35.022 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-08-11T15:35:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:35:35.136 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T15:35:35.404 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T15:35:37.267 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T15:35:37.415 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T15:35:37.700 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T15:35:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T15:35:38.419 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T15:35:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T15:35:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T15:35:39.694 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T15:35:39.990 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 115/134 2023-08-11T15:35:40.177 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T15:35:40.375 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T15:35:41.252 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-11T15:35:41.537 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T15:35:41.601 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-11T15:35:41.720 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T15:35:41.771 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 119/134 2023-08-11T15:35:41.834 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 102/134 2023-08-11T15:35:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T15:35:42.075 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 103/134 2023-08-11T15:35:42.156 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 121/134 2023-08-11T15:35:42.277 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T15:35:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T15:35:42.559 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T15:35:42.747 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T15:35:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T15:35:44.136 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T15:35:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T15:35:44.513 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:35:44.685 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 107/134 2023-08-11T15:35:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:35:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:35:44.904 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T15:35:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T15:35:45.019 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 124/134 2023-08-11T15:35:45.168 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T15:35:45.198 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T15:35:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 125/134 2023-08-11T15:35:45.625 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 126/134 2023-08-11T15:35:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-11T15:35:46.076 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T15:35:46.085 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T15:35:46.290 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-11T15:35:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T15:35:46.608 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T15:35:46.679 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el8.x86_64 110/134 2023-08-11T15:35:46.818 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T15:35:46.831 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T15:35:46.831 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:35:46.859 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T15:35:46.923 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T15:35:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:35:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T15:35:47.069 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T15:35:47.153 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T15:35:47.294 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T15:35:47.299 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T15:35:47.566 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:35:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T15:35:47.919 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T15:35:48.091 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:35:48.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el8.x86_64 1/134 2023-08-11T15:35:48.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2/134 2023-08-11T15:35:48.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 3/134 2023-08-11T15:35:48.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 4/134 2023-08-11T15:35:48.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/134 2023-08-11T15:35:48.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:35:48.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 10/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 11/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 12/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 13/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 14/134 2023-08-11T15:35:48.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 15/134 2023-08-11T15:35:48.094 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 16/134 2023-08-11T15:35:48.094 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 17/134 2023-08-11T15:35:48.094 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 18/134 2023-08-11T15:35:48.094 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 19/134 2023-08-11T15:35:48.094 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/134 2023-08-11T15:35:48.094 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 21/134 2023-08-11T15:35:48.094 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 22/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 23/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 24/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 25/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 26/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 27/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 28/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/134 2023-08-11T15:35:48.095 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 30/134 2023-08-11T15:35:48.096 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 31/134 2023-08-11T15:35:48.096 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/134 2023-08-11T15:35:48.096 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/134 2023-08-11T15:35:48.096 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 34/134 2023-08-11T15:35:48.096 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/134 2023-08-11T15:35:48.096 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 36/134 2023-08-11T15:35:48.096 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el8.noarch 37/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 38/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-11T15:35:48.097 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-11T15:35:48.098 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-11T15:35:48.099 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-11T15:35:48.100 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-11T15:35:48.100 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-11T15:35:48.100 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-11T15:35:48.100 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-11T15:35:48.100 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-11T15:35:48.100 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-11T15:35:48.100 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-11T15:35:48.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-11T15:35:48.102 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-11T15:35:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-11T15:35:48.104 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-11T15:35:48.104 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-11T15:35:48.104 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-11T15:35:48.104 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-11T15:35:48.104 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-11T15:35:48.104 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-11T15:35:48.104 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-11T15:35:48.105 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-11T15:35:48.106 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-11T15:35:48.106 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-11T15:35:48.106 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-11T15:35:48.106 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-11T15:35:48.106 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-11T15:35:48.106 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-11T15:35:48.106 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-11T15:35:48.107 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el8.x86_64 129/134 2023-08-11T15:35:48.108 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-11T15:35:48.109 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 131/134 2023-08-11T15:35:48.109 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:35:48.109 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-11T15:35:50.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:35:50.469 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:53 ago on Fri 11 Aug 2023 03:33:57 PM UTC. 2023-08-11T15:35:53.002 INFO:teuthology.orchestra.run.smithi192.stdout:Installed products updated. 2023-08-11T15:35:53.218 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:35:53.218 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2023-08-11T15:35:53.218 INFO:teuthology.orchestra.run.smithi192.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-11T15:35:53.218 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.218 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.219 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-08-11T15:35:53.219 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-11T15:35:53.219 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.1-7.el8.noarch 2023-08-11T15:35:53.219 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.0.4-10.el8.x86_64 2023-08-11T15:35:53.219 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.219 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.219 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.220 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.220 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.220 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.220 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.220 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.220 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.220 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.221 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.221 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.221 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.221 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.221 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.221 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.221 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-11T15:35:53.222 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.1-9.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-3.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.223 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.224 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.224 INFO:teuthology.orchestra.run.smithi192.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-11T15:35:53.224 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-11T15:35:53.224 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-11T15:35:53.224 INFO:teuthology.orchestra.run.smithi192.stdout: libpq-13.5-1.el8.x86_64 2023-08-11T15:35:53.224 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-11T15:35:53.224 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-11T15:35:53.225 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.225 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.225 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-11T15:35:53.225 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.225 INFO:teuthology.orchestra.run.smithi192.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-11T15:35:53.225 INFO:teuthology.orchestra.run.smithi192.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-11T15:35:53.225 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-11T15:35:53.226 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-11T15:35:53.227 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-11T15:35:53.227 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-11T15:35:53.227 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-11T15:35:53.227 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:35:53.227 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:35:53.227 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:35:53.227 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-11T15:35:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-11T15:35:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-11T15:35:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-11T15:35:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-11T15:35:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.229 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-11T15:35:53.229 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-11T15:35:53.229 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-11T15:35:53.229 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-11T15:35:53.229 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-11T15:35:53.229 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-11T15:35:53.229 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-11T15:35:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-11T15:35:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-2.6-1.el8.noarch 2023-08-11T15:35:53.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-11T15:35:53.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-11T15:35:53.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-11T15:35:53.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-11T15:35:53.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-11T15:35:53.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-11T15:35:53.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-11T15:35:53.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-11T15:35:53.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-11T15:35:53.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-11T15:35:53.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-11T15:35:53.234 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-11T15:35:53.235 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-11T15:35:53.235 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-11T15:35:53.235 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.235 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.235 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:35:53.235 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-11T15:35:53.235 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-11T15:35:53.236 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-11T15:35:53.236 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-11T15:35:53.236 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-11T15:35:53.237 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-11T15:35:53.237 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:35:53.238 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-08-11T15:35:53.329 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T15:35:53.393 DEBUG:teuthology.parallel:result is None 2023-08-11T15:35:54.654 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 115/134 2023-08-11T15:35:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-08-11T15:35:55.267 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 117/134 2023-08-11T15:35:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-08-11T15:35:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-08-11T15:35:55.937 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-08-11T15:35:56.315 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-08-11T15:35:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 121/134 2023-08-11T15:35:56.644 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T15:36:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:36:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:36:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T15:36:00.735 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 124/134 2023-08-11T15:36:00.961 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 125/134 2023-08-11T15:36:01.346 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 126/134 2023-08-11T15:36:01.706 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-11T15:36:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-08-11T15:36:02.000 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-08-11T15:36:02.233 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-08-11T15:36:02.262 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-08-11T15:36:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-11T15:36:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-11T15:36:02.662 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-08-11T15:36:02.662 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:36:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:36:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T15:36:03.208 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T15:36:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:36:03.656 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:36:03.657 INFO:teuthology.orchestra.run.smithi032.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-11T15:36:03.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:36:04.276 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el8.x86_64 1/134 2023-08-11T15:36:04.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2/134 2023-08-11T15:36:04.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 3/134 2023-08-11T15:36:04.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 4/134 2023-08-11T15:36:04.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/134 2023-08-11T15:36:04.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T15:36:04.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T15:36:04.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 10/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 11/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 12/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 13/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 14/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 15/134 2023-08-11T15:36:04.278 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 16/134 2023-08-11T15:36:04.279 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 17/134 2023-08-11T15:36:04.279 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 18/134 2023-08-11T15:36:04.279 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 19/134 2023-08-11T15:36:04.279 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/134 2023-08-11T15:36:04.279 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 21/134 2023-08-11T15:36:04.279 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 22/134 2023-08-11T15:36:04.279 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 23/134 2023-08-11T15:36:04.280 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 24/134 2023-08-11T15:36:04.280 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 25/134 2023-08-11T15:36:04.280 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 26/134 2023-08-11T15:36:04.280 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 27/134 2023-08-11T15:36:04.280 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 28/134 2023-08-11T15:36:04.280 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/134 2023-08-11T15:36:04.287 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 30/134 2023-08-11T15:36:04.287 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 31/134 2023-08-11T15:36:04.287 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/134 2023-08-11T15:36:04.287 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/134 2023-08-11T15:36:04.288 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 34/134 2023-08-11T15:36:04.288 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/134 2023-08-11T15:36:04.288 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 36/134 2023-08-11T15:36:04.288 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el8.noarch 37/134 2023-08-11T15:36:04.288 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 38/134 2023-08-11T15:36:04.288 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-11T15:36:04.288 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-11T15:36:04.289 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-11T15:36:04.290 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-11T15:36:04.290 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-11T15:36:04.290 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-11T15:36:04.290 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-11T15:36:04.290 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-11T15:36:04.290 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-11T15:36:04.290 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-11T15:36:04.291 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-11T15:36:04.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-08-11T15:36:04.293 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-08-11T15:36:04.294 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-08-11T15:36:04.294 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-08-11T15:36:04.294 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-08-11T15:36:04.294 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-08-11T15:36:04.294 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-08-11T15:36:04.294 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-08-11T15:36:04.294 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-08-11T15:36:04.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-08-11T15:36:04.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-08-11T15:36:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-08-11T15:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-08-11T15:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-08-11T15:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-08-11T15:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-08-11T15:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-08-11T15:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-08-11T15:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-08-11T15:36:04.299 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el8.x86_64 129/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 131/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T15:36:04.300 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-11T15:36:06.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T15:36:06.965 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:10 ago on Fri 11 Aug 2023 03:33:56 PM UTC. 2023-08-11T15:36:09.850 INFO:teuthology.orchestra.run.smithi032.stdout:Installed products updated. 2023-08-11T15:36:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:36:10.092 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2023-08-11T15:36:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-11T15:36:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.093 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-08-11T15:36:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-11T15:36:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.1-7.el8.noarch 2023-08-11T15:36:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.0.4-10.el8.x86_64 2023-08-11T15:36:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T15:36:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-11T15:36:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.1-9.el8.x86_64 2023-08-11T15:36:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-11T15:36:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: jq-1.6-3.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-11T15:36:10.099 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-11T15:36:10.099 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-11T15:36:10.099 INFO:teuthology.orchestra.run.smithi032.stdout: libpq-13.5-1.el8.x86_64 2023-08-11T15:36:10.099 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-11T15:36:10.099 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-11T15:36:10.099 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.099 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.100 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-11T15:36:10.100 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.100 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-11T15:36:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-11T15:36:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-11T15:36:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-11T15:36:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-11T15:36:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-11T15:36:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-11T15:36:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:36:10.102 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:36:10.103 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T15:36:10.103 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-11T15:36:10.103 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-11T15:36:10.103 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-11T15:36:10.103 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-11T15:36:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-11T15:36:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-11T15:36:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-11T15:36:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-11T15:36:10.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-11T15:36:10.106 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-11T15:36:10.106 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-11T15:36:10.106 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-11T15:36:10.106 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-11T15:36:10.106 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-11T15:36:10.107 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-11T15:36:10.107 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-11T15:36:10.107 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-11T15:36:10.107 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-11T15:36:10.108 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-11T15:36:10.108 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-11T15:36:10.108 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-2.6-1.el8.noarch 2023-08-11T15:36:10.108 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-11T15:36:10.108 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-11T15:36:10.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-11T15:36:10.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-11T15:36:10.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-11T15:36:10.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-11T15:36:10.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-11T15:36:10.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-11T15:36:10.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-11T15:36:10.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-11T15:36:10.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-11T15:36:10.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-11T15:36:10.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-11T15:36:10.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-11T15:36:10.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-11T15:36:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-11T15:36:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-11T15:36:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-11T15:36:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-11T15:36:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-11T15:36:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-11T15:36:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.114 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.114 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T15:36:10.114 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-11T15:36:10.114 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-11T15:36:10.115 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-11T15:36:10.115 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-11T15:36:10.115 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-11T15:36:10.115 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-11T15:36:10.116 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:36:10.116 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-08-11T15:36:10.455 DEBUG:teuthology.parallel:result is None 2023-08-11T15:36:10.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:36:10.598 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T15:36:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:18.0.0-5437.g92684807.el8 2023-08-11T15:36:10.659 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el8 2023-08-11T15:36:10.659 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T15:36:10.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:36:10.873 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T15:36:10.903 INFO:teuthology.orchestra.run.smithi134.stdout:18.0.0-5437.g92684807.el8 2023-08-11T15:36:10.903 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el8 2023-08-11T15:36:10.903 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T15:36:10.904 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:36:11.117 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T15:36:11.146 INFO:teuthology.orchestra.run.smithi192.stdout:18.0.0-5437.g92684807.el8 2023-08-11T15:36:11.146 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el8 2023-08-11T15:36:11.147 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T15:36:11.147 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-11T15:36:11.148 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:36:11.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T15:36:11.243 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:36:11.243 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T15:36:11.290 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:36:11.291 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T15:36:11.338 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-11T15:36:11.338 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:36:11.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T15:36:11.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T15:36:11.523 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:36:11.523 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T15:36:11.555 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T15:36:11.625 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:36:11.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T15:36:11.657 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T15:36:11.729 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-11T15:36:11.729 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:36:11.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T15:36:11.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T15:36:11.913 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:36:11.914 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T15:36:11.945 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T15:36:12.014 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:36:12.015 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T15:36:12.046 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T15:36:12.116 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-11T15:36:12.116 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:36:12.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T15:36:12.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T15:36:12.295 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:36:12.296 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T15:36:12.327 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T15:36:12.397 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:36:12.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T15:36:12.430 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T15:36:12.500 INFO:teuthology.run_tasks:Running task cephadm... 2023-08-11T15:36:12.599 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-08-11T15:36:12.600 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:36:12.612 INFO:tasks.cephadm:Cluster fsid is cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:36:12.613 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-08-11T15:36:12.613 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.32', 'mon.b': '172.21.15.134', 'mon.c': '172.21.15.192'} 2023-08-11T15:36:12.613 INFO:tasks.cephadm:First mon is mon.a on smithi032 2023-08-11T15:36:12.614 INFO:tasks.cephadm:First mgr is x 2023-08-11T15:36:12.614 INFO:tasks.cephadm:Normalizing hostnames... 2023-08-11T15:36:12.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname $(hostname -s) 2023-08-11T15:36:12.687 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2023-08-11T15:36:12.719 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2023-08-11T15:36:12.752 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-08-11T15:36:12.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:36:12.895 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71982/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-5437-g92684807', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-5437.g92684807'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-08-10 16:30:02.468413', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Aug1-b4', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-08-11T15:36:13.125 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-2023Aug1-b4, sha1 92684807f016ce045ac66e37fdde6f8c47857bcc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:36:13.126 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm 2023-08-11T15:36:13.126 DEBUG:teuthology.orchestra.run.smithi032:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:13.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-rw-r--. 1 ubuntu ubuntu 698557 Aug 11 15:36 /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:13.741 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:14.496 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-rw-r--. 1 ubuntu ubuntu 698557 Aug 11 15:36 /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:14.496 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:14.869 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-rw-r--. 1 ubuntu ubuntu 698557 Aug 11 15:36 /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:14.870 DEBUG:teuthology.orchestra.run.smithi032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:14.922 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:14.943 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-11T15:36:14.978 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc on all hosts... 2023-08-11T15:36:14.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc pull 2023-08-11T15:36:14.983 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc pull 2023-08-11T15:36:14.986 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc pull 2023-08-11T15:36:16.333 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T15:36:16.702 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T15:36:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T15:36:46.617 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-08-11T15:36:46.617 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:36:46.617 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c", 2023-08-11T15:36:46.617 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2023-08-11T15:36:46.617 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75a49d67f8e63bcfe7c72420c0bd76f2f6be8c4f86dd373c0bd2e3def3a2e2f3" 2023-08-11T15:36:46.618 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2023-08-11T15:36:46.618 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-08-11T15:36:46.708 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-08-11T15:36:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:36:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c", 2023-08-11T15:36:46.708 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2023-08-11T15:36:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75a49d67f8e63bcfe7c72420c0bd76f2f6be8c4f86dd373c0bd2e3def3a2e2f3" 2023-08-11T15:36:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2023-08-11T15:36:46.709 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-08-11T15:36:46.816 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:36:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:36:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "image_id": "a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c", 2023-08-11T15:36:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "repo_digests": [ 2023-08-11T15:36:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75a49d67f8e63bcfe7c72420c0bd76f2f6be8c4f86dd373c0bd2e3def3a2e2f3" 2023-08-11T15:36:46.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:36:46.817 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:36:46.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph 2023-08-11T15:36:46.915 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2023-08-11T15:36:46.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2023-08-11T15:36:46.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /etc/ceph 2023-08-11T15:36:47.051 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2023-08-11T15:36:47.079 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2023-08-11T15:36:47.109 INFO:tasks.cephadm:Writing seed config... 2023-08-11T15:36:47.110 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-08-11T15:36:47.110 INFO:tasks.cephadm: override: [client] debug client = 20 2023-08-11T15:36:47.110 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-08-11T15:36:47.111 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-08-11T15:36:47.111 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-08-11T15:36:47.111 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-08-11T15:36:47.111 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-08-11T15:36:47.111 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-08-11T15:36:47.112 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-08-11T15:36:47.112 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-08-11T15:36:47.112 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-08-11T15:36:47.112 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-08-11T15:36:47.113 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-08-11T15:36:47.113 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-08-11T15:36:47.113 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-08-11T15:36:47.113 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-08-11T15:36:47.113 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-08-11T15:36:47.114 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-08-11T15:36:47.114 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-08-11T15:36:47.114 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-08-11T15:36:47.114 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-08-11T15:36:47.114 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-08-11T15:36:47.115 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-08-11T15:36:47.115 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-08-11T15:36:47.115 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-08-11T15:36:47.115 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-08-11T15:36:47.115 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-08-11T15:36:47.116 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-08-11T15:36:47.116 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-08-11T15:36:47.116 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-08-11T15:36:47.117 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-08-11T15:36:47.117 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-08-11T15:36:47.117 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-08-11T15:36:47.117 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-08-11T15:36:47.117 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-08-11T15:36:47.118 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-08-11T15:36:47.118 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-08-11T15:36:47.118 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-08-11T15:36:47.118 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-08-11T15:36:47.118 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-08-11T15:36:47.119 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-08-11T15:36:47.119 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-08-11T15:36:47.120 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-08-11T15:36:47.120 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:36:47.121 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-08-11T15:36:47.164 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = cd36a838-385c-11ee-9b37-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-08-11T15:36:47.164 DEBUG:teuthology.orchestra.run.smithi032:mon.a> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a.service 2023-08-11T15:36:47.208 DEBUG:teuthology.orchestra.run.smithi032:mgr.x> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.x.service 2023-08-11T15:36:47.253 INFO:tasks.cephadm:Bootstrapping... 2023-08-11T15:36:47.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc -v bootstrap --fsid cd36a838-385c-11ee-9b37-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.32 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-08-11T15:36:47.278 INFO:journalctl@ceph.mon.a.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:36:47.323 INFO:journalctl@ceph.mgr.x.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:36:47.487 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-08-11T15:36:47.488 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc', '-v', 'bootstrap', '--fsid', 'cd36a838-385c-11ee-9b37-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.32', '--skip-admin-label'] 2023-08-11T15:36:47.661 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.1.1 2023-08-11T15:36:47.661 INFO:teuthology.orchestra.run.smithi032.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-08-11T15:36:47.662 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying podman|docker is present... 2023-08-11T15:36:47.834 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.1.1 2023-08-11T15:36:47.835 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying lvm2 is present... 2023-08-11T15:36:47.835 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying time synchronization is in place... 2023-08-11T15:36:47.849 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-11T15:36:47.849 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-11T15:36:47.862 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-11T15:36:47.863 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-08-11T15:36:47.876 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2023-08-11T15:36:47.889 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2023-08-11T15:36:47.889 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2023-08-11T15:36:47.889 INFO:teuthology.orchestra.run.smithi032.stdout:Repeating the final host check... 2023-08-11T15:36:48.059 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.1.1 2023-08-11T15:36:48.059 INFO:teuthology.orchestra.run.smithi032.stdout:podman (/bin/podman) version 4.1.1 is present 2023-08-11T15:36:48.060 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl is present 2023-08-11T15:36:48.060 INFO:teuthology.orchestra.run.smithi032.stdout:lvcreate is present 2023-08-11T15:36:48.072 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-11T15:36:48.073 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-11T15:36:48.085 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-11T15:36:48.086 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-08-11T15:36:48.099 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2023-08-11T15:36:48.113 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2023-08-11T15:36:48.113 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2023-08-11T15:36:48.113 INFO:teuthology.orchestra.run.smithi032.stdout:Host looks OK 2023-08-11T15:36:48.113 INFO:teuthology.orchestra.run.smithi032.stdout:Cluster fsid: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:36:48.113 INFO:teuthology.orchestra.run.smithi032.stdout:Acquiring lock 139690314934760 on /run/cephadm/cd36a838-385c-11ee-9b37-001a4aab830c.lock 2023-08-11T15:36:48.114 INFO:teuthology.orchestra.run.smithi032.stdout:Lock 139690314934760 acquired on /run/cephadm/cd36a838-385c-11ee-9b37-001a4aab830c.lock 2023-08-11T15:36:48.114 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 3300 ... 2023-08-11T15:36:48.114 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 6789 ... 2023-08-11T15:36:48.115 INFO:teuthology.orchestra.run.smithi032.stdout:Base mon IP(s) is [172.21.15.32:3300, 172.21.15.32:6789], mon addrv is [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2023-08-11T15:36:48.122 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-08-11T15:36:48.122 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.32 metric 100 2023-08-11T15:36:48.128 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-08-11T15:36:48.128 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-08-11T15:36:48.129 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2023-08-11T15:36:48.135 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-08-11T15:36:48.135 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-08-11T15:36:48.135 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-11T15:36:48.136 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-08-11T15:36:48.136 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-08-11T15:36:48.136 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-11T15:36:48.136 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2023-08-11T15:36:48.137 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2023-08-11T15:36:48.137 INFO:teuthology.orchestra.run.smithi032.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-08-11T15:36:48.137 INFO:teuthology.orchestra.run.smithi032.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-08-11T15:36:48.138 INFO:teuthology.orchestra.run.smithi032.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T15:36:48.612 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c 2023-08-11T15:36:48.613 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T15:36:48.613 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Getting image source signatures 2023-08-11T15:36:48.613 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:1466c0ce567d8cab08280e456a5ef839cd0bf60a0cec2e3199abbfddb92a2725 2023-08-11T15:36:48.613 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-08-11T15:36:48.614 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying config sha256:a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c 2023-08-11T15:36:48.614 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Writing manifest to image destination 2023-08-11T15:36:48.614 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Storing signatures 2023-08-11T15:36:49.485 INFO:teuthology.orchestra.run.smithi032.stdout:ceph: stdout ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev) 2023-08-11T15:36:49.486 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph version: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev) 2023-08-11T15:36:49.486 INFO:teuthology.orchestra.run.smithi032.stdout:Extracting ceph user uid/gid from container image... 2023-08-11T15:36:50.264 INFO:teuthology.orchestra.run.smithi032.stdout:stat: stdout 167 167 2023-08-11T15:36:50.264 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial keys... 2023-08-11T15:36:51.038 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCSVdZkQPnMJxAAF6+WvfURTbd+gHT8SGvETQ== 2023-08-11T15:36:51.816 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCTVdZkVS1kGBAAJ31tbHz0rqef8okO/8GTog== 2023-08-11T15:36:52.690 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCUVdZkL939DRAAFW/MjMQUemGgAl0aoc76vQ== 2023-08-11T15:36:52.691 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial monmap... 2023-08-11T15:36:53.465 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-11T15:36:53.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-08-11T15:36:53.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:36:53.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-11T15:36:53.466 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool for a [v2:172.21.15.32:3300,v1:172.21.15.32:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-11T15:36:53.466 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2023-08-11T15:36:53.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: set fsid to cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:36:53.466 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-11T15:36:53.467 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:36:53.467 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mon... 2023-08-11T15:36:54.246 INFO:teuthology.orchestra.run.smithi032.stdout:create mon.a on 2023-08-11T15:36:54.503 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-08-11T15:36:54.723 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-08-11T15:36:54.989 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-cd36a838-385c-11ee-9b37-001a4aab830c.target → /etc/systemd/system/ceph-cd36a838-385c-11ee-9b37-001a4aab830c.target. 2023-08-11T15:36:54.989 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-cd36a838-385c-11ee-9b37-001a4aab830c.target → /etc/systemd/system/ceph-cd36a838-385c-11ee-9b37-001a4aab830c.target. 2023-08-11T15:36:55.377 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a 2023-08-11T15:36:55.378 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a.service: Unit ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a.service not loaded. 2023-08-11T15:36:55.582 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-cd36a838-385c-11ee-9b37-001a4aab830c.target.wants/ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-cd36a838-385c-11ee-9b37-001a4aab830c@.service. 2023-08-11T15:36:55.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:55 smithi032 systemd[1]: Starting Ceph mon.a for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:36:56.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:56 smithi032 podman[96556]: 2023-08-11T15:36:56.371 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:56 smithi032 bash[96556]: 1003f860ca6222752ef6114179cb7c13c463ba12b9400ebc005573384b08a07b 2023-08-11T15:36:56.403 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T15:36:56.403 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-08-11T15:36:56.419 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T15:36:56.419 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-08-11T15:36:56.419 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-08-11T15:36:56.420 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2023-08-11T15:36:56.420 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon to start... 2023-08-11T15:36:56.420 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon... 2023-08-11T15:36:56.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:56 smithi032 ceph-mon[96596]: mkfs cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:36:56.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:56 smithi032 ceph-mon[96596]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-11T15:36:56.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:56 smithi032 systemd[1]: Started Ceph mon.a for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:36:57.803 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout cluster: 2023-08-11T15:36:57.803 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout id: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:36:57.803 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-08-11T15:36:57.803 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:36:57.803 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout services: 2023-08-11T15:36:57.804 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.07633s) 2023-08-11T15:36:57.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:57 smithi032 ceph-mon[96596]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-11T15:36:57.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:57 smithi032 ceph-mon[96596]: monmap e1: 1 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} 2023-08-11T15:36:57.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:57 smithi032 ceph-mon[96596]: fsmap 2023-08-11T15:36:57.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:57 smithi032 ceph-mon[96596]: osdmap e1: 0 total, 0 up, 0 in 2023-08-11T15:36:57.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:57 smithi032 ceph-mon[96596]: mgrmap e1: no daemons active 2023-08-11T15:36:57.810 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-08-11T15:36:57.810 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-08-11T15:36:57.811 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:36:57.811 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout data: 2023-08-11T15:36:57.811 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-08-11T15:36:57.811 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-08-11T15:36:57.811 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-08-11T15:36:57.812 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pgs: 2023-08-11T15:36:57.812 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:36:57.812 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.355+0000 7f32e85eb700 1 Processor -- start 2023-08-11T15:36:57.812 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.355+0000 7f32e85eb700 1 -- start start 2023-08-11T15:36:57.812 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.355+0000 7f32e85eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:36:57.813 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.355+0000 7f32e85eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e007a890 con 0x7f32e007be60 2023-08-11T15:36:57.813 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.356+0000 7f32e6387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:36:57.813 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.356+0000 7f32e6387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35178/0 (socket says 172.21.15.32:35178) 2023-08-11T15:36:57.813 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.356+0000 7f32e6387700 1 -- 172.21.15.32:0/2497051394 learned_addr learned my addr 172.21.15.32:0/2497051394 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:36:57.813 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.357+0000 7f32e6387700 1 -- 172.21.15.32:0/2497051394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e007a9d0 con 0x7f32e007be60 2023-08-11T15:36:57.813 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.357+0000 7f32e6387700 1 --2- 172.21.15.32:0/2497051394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e007a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0098a0 tx=0x7f32dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e28468bf3bfc1b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:36:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.357+0000 7f32e5385700 1 -- 172.21.15.32:0/2497051394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc004160 con 0x7f32e007be60 2023-08-11T15:36:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.357+0000 7f32e5385700 1 -- 172.21.15.32:0/2497051394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0042c0 con 0x7f32e007be60 2023-08-11T15:36:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.357+0000 7f32e5385700 1 -- 172.21.15.32:0/2497051394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc0045a0 con 0x7f32e007be60 2023-08-11T15:36:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.358+0000 7f32e85eb700 1 -- 172.21.15.32:0/2497051394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 msgr2=0x7f32e007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:36:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.358+0000 7f32e85eb700 1 --2- 172.21.15.32:0/2497051394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e007a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0098a0 tx=0x7f32dc009bb0 comp rx=0 tx=0).stop 2023-08-11T15:36:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.358+0000 7f32e85eb700 1 -- 172.21.15.32:0/2497051394 shutdown_connections 2023-08-11T15:36:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.358+0000 7f32e85eb700 1 --2- 172.21.15.32:0/2497051394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e007a2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:36:57.815 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.358+0000 7f32e85eb700 1 -- 172.21.15.32:0/2497051394 >> 172.21.15.32:0/2497051394 conn(0x7f32e0101610 msgr2=0x7f32e0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:36:57.815 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.358+0000 7f32e85eb700 1 -- 172.21.15.32:0/2497051394 shutdown_connections 2023-08-11T15:36:57.815 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.358+0000 7f32e85eb700 1 -- 172.21.15.32:0/2497051394 wait complete. 2023-08-11T15:36:57.815 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.359+0000 7f32e85eb700 1 Processor -- start 2023-08-11T15:36:57.815 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.359+0000 7f32e85eb700 1 -- start start 2023-08-11T15:36:57.816 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.359+0000 7f32e85eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:36:57.816 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.359+0000 7f32e85eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e019bc90 con 0x7f32e007be60 2023-08-11T15:36:57.816 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.360+0000 7f32e6387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:36:57.816 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.360+0000 7f32e6387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35192/0 (socket says 172.21.15.32:35192) 2023-08-11T15:36:57.816 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.360+0000 7f32e6387700 1 -- 172.21.15.32:0/168589432 learned_addr learned my addr 172.21.15.32:0/168589432 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:36:57.816 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.360+0000 7f32e6387700 1 -- 172.21.15.32:0/168589432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc009550 con 0x7f32e007be60 2023-08-11T15:36:57.817 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32e6387700 1 --2- 172.21.15.32:0/168589432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e019b750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f32e007b330 tx=0x7f32dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:36:57.817 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32d77fe700 1 -- 172.21.15.32:0/168589432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc011630 con 0x7f32e007be60 2023-08-11T15:36:57.817 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32d77fe700 1 -- 172.21.15.32:0/168589432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc011790 con 0x7f32e007be60 2023-08-11T15:36:57.817 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32d77fe700 1 -- 172.21.15.32:0/168589432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc011a30 con 0x7f32e007be60 2023-08-11T15:36:57.817 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e019be90 con 0x7f32e007be60 2023-08-11T15:36:57.817 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e019c2b0 con 0x7f32e007be60 2023-08-11T15:36:57.818 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32d77fe700 1 -- 172.21.15.32:0/168589432 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f32dc022070 con 0x7f32e007be60 2023-08-11T15:36:57.818 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.361+0000 7f32d77fe700 1 -- 172.21.15.32:0/168589432 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f32dc01c450 con 0x7f32e007be60 2023-08-11T15:36:57.818 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.362+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e0195600 con 0x7f32e007be60 2023-08-11T15:36:57.818 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.365+0000 7f32d77fe700 1 -- 172.21.15.32:0/168589432 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f32dc01b450 con 0x7f32e007be60 2023-08-11T15:36:57.818 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.441+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f32e007ad40 con 0x7f32e007be60 2023-08-11T15:36:57.818 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.441+0000 7f32d77fe700 1 -- 172.21.15.32:0/168589432 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f32dc01b5f0 con 0x7f32e007be60 2023-08-11T15:36:57.819 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.442+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 msgr2=0x7f32e019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:36:57.819 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.442+0000 7f32e85eb700 1 --2- 172.21.15.32:0/168589432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e019b750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f32e007b330 tx=0x7f32dc004750 comp rx=0 tx=0).stop 2023-08-11T15:36:57.819 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.442+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 shutdown_connections 2023-08-11T15:36:57.819 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.442+0000 7f32e85eb700 1 --2- 172.21.15.32:0/168589432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e007be60 0x7f32e019b750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:36:57.819 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.442+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 >> 172.21.15.32:0/168589432 conn(0x7f32e0101610 msgr2=0x7f32e0103020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:36:57.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.443+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 shutdown_connections 2023-08-11T15:36:57.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:57.443+0000 7f32e85eb700 1 -- 172.21.15.32:0/168589432 wait complete. 2023-08-11T15:36:57.820 INFO:teuthology.orchestra.run.smithi032.stdout:mon is available 2023-08-11T15:36:57.820 INFO:teuthology.orchestra.run.smithi032.stdout:Assimilating anything we can from ceph.conf... 2023-08-11T15:36:58.651 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:58 smithi032 ceph-mon[96596]: from='client.? 172.21.15.32:0/168589432' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-08-11T15:36:58.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:36:58.891 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2023-08-11T15:36:58.891 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:36:58.891 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2023-08-11T15:36:58.891 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-11T15:36:58.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-11T15:36:58.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-11T15:36:58.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-11T15:36:58.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:36:58.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-11T15:36:58.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-11T15:36:58.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:36:58.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2023-08-11T15:36:58.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-11T15:36:58.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-11T15:36:58.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-11T15:36:58.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-11T15:36:58.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-11T15:36:58.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.361+0000 7f472a7de700 1 Processor -- start 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.361+0000 7f472a7de700 1 -- start start 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.361+0000 7f472a7de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.361+0000 7f472a7de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472407a8f0 con 0x7f472407bec0 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.362+0000 7f47297dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.362+0000 7f47297dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35198/0 (socket says 172.21.15.32:35198) 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.362+0000 7f47297dc700 1 -- 172.21.15.32:0/3949183202 learned_addr learned my addr 172.21.15.32:0/3949183202 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.362+0000 7f47297dc700 1 -- 172.21.15.32:0/3949183202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472407aa30 con 0x7f472407bec0 2023-08-11T15:36:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.362+0000 7f47297dc700 1 --2- 172.21.15.32:0/3949183202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472407a320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47200098a0 tx=0x7f4720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2566b2e05d6ed8a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:36:58.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f471bfff700 1 -- 172.21.15.32:0/3949183202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4720004160 con 0x7f472407bec0 2023-08-11T15:36:58.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f471bfff700 1 -- 172.21.15.32:0/3949183202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47200042c0 con 0x7f472407bec0 2023-08-11T15:36:58.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f471bfff700 1 -- 172.21.15.32:0/3949183202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47200045a0 con 0x7f472407bec0 2023-08-11T15:36:58.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f472a7de700 1 -- 172.21.15.32:0/3949183202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 msgr2=0x7f472407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:36:58.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f472a7de700 1 --2- 172.21.15.32:0/3949183202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472407a320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47200098a0 tx=0x7f4720009bb0 comp rx=0 tx=0).stop 2023-08-11T15:36:58.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f472a7de700 1 -- 172.21.15.32:0/3949183202 shutdown_connections 2023-08-11T15:36:58.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f472a7de700 1 --2- 172.21.15.32:0/3949183202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472407a320 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:36:58.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.363+0000 7f472a7de700 1 -- 172.21.15.32:0/3949183202 >> 172.21.15.32:0/3949183202 conn(0x7f4724101440 msgr2=0x7f4724103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:36:58.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.364+0000 7f472a7de700 1 -- 172.21.15.32:0/3949183202 shutdown_connections 2023-08-11T15:36:58.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.364+0000 7f472a7de700 1 -- 172.21.15.32:0/3949183202 wait complete. 2023-08-11T15:36:58.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.365+0000 7f472a7de700 1 Processor -- start 2023-08-11T15:36:58.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.365+0000 7f472a7de700 1 -- start start 2023-08-11T15:36:58.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.365+0000 7f472a7de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:36:58.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.365+0000 7f472a7de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472419bb00 con 0x7f472407bec0 2023-08-11T15:36:58.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.365+0000 7f47297dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:36:58.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f47297dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35204/0 (socket says 172.21.15.32:35204) 2023-08-11T15:36:58.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f47297dc700 1 -- 172.21.15.32:0/3940018273 learned_addr learned my addr 172.21.15.32:0/3940018273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:36:58.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f47297dc700 1 -- 172.21.15.32:0/3940018273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4720009550 con 0x7f472407bec0 2023-08-11T15:36:58.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f47297dc700 1 --2- 172.21.15.32:0/3940018273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472419b5c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4720004000 tx=0x7f4720004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:36:58.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4720011630 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720011790 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472419bd00 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.366+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4720011a30 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.367+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472419c120 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.367+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4720022070 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.367+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f472001c450 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.368+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4724195440 con 0x7f472407bec0 2023-08-11T15:36:58.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.370+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f472001b450 con 0x7f472407bec0 2023-08-11T15:36:58.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.444+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f472407b020 con 0x7f472407bec0 2023-08-11T15:36:58.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.449+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f472001b5f0 con 0x7f472407bec0 2023-08-11T15:36:58.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.449+0000 7f471a7fc700 1 -- 172.21.15.32:0/3940018273 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f4720036540 con 0x7f472407bec0 2023-08-11T15:36:58.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.450+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 msgr2=0x7f472419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:36:58.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.450+0000 7f472a7de700 1 --2- 172.21.15.32:0/3940018273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472419b5c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4720004000 tx=0x7f4720004750 comp rx=0 tx=0).stop 2023-08-11T15:36:58.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.451+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 shutdown_connections 2023-08-11T15:36:58.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.451+0000 7f472a7de700 1 --2- 172.21.15.32:0/3940018273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f472407bec0 0x7f472419b5c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:36:58.900 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.451+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 >> 172.21.15.32:0/3940018273 conn(0x7f4724101440 msgr2=0x7f4724102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:36:58.900 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.451+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 shutdown_connections 2023-08-11T15:36:58.900 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:58.451+0000 7f472a7de700 1 -- 172.21.15.32:0/3940018273 wait complete. 2023-08-11T15:36:58.900 INFO:teuthology.orchestra.run.smithi032.stdout:Generating new minimal ceph.conf... 2023-08-11T15:36:59.504 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:59 smithi032 ceph-mon[96596]: from='client.? 172.21.15.32:0/3940018273' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-11T15:36:59.504 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:59 smithi032 ceph-mon[96596]: from='client.? 172.21.15.32:0/3940018273' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-08-11T15:36:59.868 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.399+0000 7f2125602700 1 Processor -- start 2023-08-11T15:36:59.869 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.399+0000 7f2125602700 1 -- start start 2023-08-11T15:36:59.869 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.399+0000 7f2125602700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:36:59.869 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.399+0000 7f2125602700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212007a890 con 0x7f212007be60 2023-08-11T15:36:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.400+0000 7f211effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:36:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.400+0000 7f211effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35218/0 (socket says 172.21.15.32:35218) 2023-08-11T15:36:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.400+0000 7f211effd700 1 -- 172.21.15.32:0/1598349609 learned_addr learned my addr 172.21.15.32:0/1598349609 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:36:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.400+0000 7f211effd700 1 -- 172.21.15.32:0/1598349609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212007a9d0 con 0x7f212007be60 2023-08-11T15:36:59.870 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.400+0000 7f211effd700 1 --2- 172.21.15.32:0/1598349609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212007a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21100098a0 tx=0x7f2110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3d4e855ce8ccd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:36:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.400+0000 7f211dffb700 1 -- 172.21.15.32:0/1598349609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2110004160 con 0x7f212007be60 2023-08-11T15:36:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f211dffb700 1 -- 172.21.15.32:0/1598349609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21100042c0 con 0x7f212007be60 2023-08-11T15:36:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f211dffb700 1 -- 172.21.15.32:0/1598349609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2110011a00 con 0x7f212007be60 2023-08-11T15:36:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f2125602700 1 -- 172.21.15.32:0/1598349609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 msgr2=0x7f212007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:36:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f2125602700 1 --2- 172.21.15.32:0/1598349609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212007a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21100098a0 tx=0x7f2110009bb0 comp rx=0 tx=0).stop 2023-08-11T15:36:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f2125602700 1 -- 172.21.15.32:0/1598349609 shutdown_connections 2023-08-11T15:36:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f2125602700 1 --2- 172.21.15.32:0/1598349609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212007a2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:36:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f2125602700 1 -- 172.21.15.32:0/1598349609 >> 172.21.15.32:0/1598349609 conn(0x7f2120101610 msgr2=0x7f2120103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:36:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f2125602700 1 -- 172.21.15.32:0/1598349609 shutdown_connections 2023-08-11T15:36:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.401+0000 7f2125602700 1 -- 172.21.15.32:0/1598349609 wait complete. 2023-08-11T15:36:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.402+0000 7f2125602700 1 Processor -- start 2023-08-11T15:36:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.402+0000 7f2125602700 1 -- start start 2023-08-11T15:36:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.403+0000 7f2125602700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:36:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.403+0000 7f2125602700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212019c530 con 0x7f212007be60 2023-08-11T15:36:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.403+0000 7f211effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:36:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.403+0000 7f211effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212019bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35234/0 (socket says 172.21.15.32:35234) 2023-08-11T15:36:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.403+0000 7f211effd700 1 -- 172.21.15.32:0/2050139404 learned_addr learned my addr 172.21.15.32:0/2050139404 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:36:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.403+0000 7f211effd700 1 -- 172.21.15.32:0/2050139404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110009550 con 0x7f212007be60 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f211effd700 1 --2- 172.21.15.32:0/2050139404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212019bff0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2110009fb0 tx=0x7f2110011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f2107fff700 1 -- 172.21.15.32:0/2050139404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2110003a40 con 0x7f212007be60 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f2107fff700 1 -- 172.21.15.32:0/2050139404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2110003ba0 con 0x7f212007be60 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f2107fff700 1 -- 172.21.15.32:0/2050139404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f211001a750 con 0x7f212007be60 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212019c730 con 0x7f212007be60 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212019cb50 con 0x7f212007be60 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f2107fff700 1 -- 172.21.15.32:0/2050139404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f211001ac80 con 0x7f212007be60 2023-08-11T15:36:59.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.404+0000 7f2107fff700 1 -- 172.21.15.32:0/2050139404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f21100193c0 con 0x7f212007be60 2023-08-11T15:36:59.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.405+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2120196040 con 0x7f212007be60 2023-08-11T15:36:59.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.408+0000 7f2107fff700 1 -- 172.21.15.32:0/2050139404 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f211002de30 con 0x7f212007be60 2023-08-11T15:36:59.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.481+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f212010c420 con 0x7f212007be60 2023-08-11T15:36:59.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.481+0000 7f2107fff700 1 -- 172.21.15.32:0/2050139404 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f2110024b60 con 0x7f212007be60 2023-08-11T15:36:59.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.482+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 msgr2=0x7f212019bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:36:59.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.482+0000 7f2125602700 1 --2- 172.21.15.32:0/2050139404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212019bff0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2110009fb0 tx=0x7f2110011ad0 comp rx=0 tx=0).stop 2023-08-11T15:36:59.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.482+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 shutdown_connections 2023-08-11T15:36:59.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.482+0000 7f2125602700 1 --2- 172.21.15.32:0/2050139404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212007be60 0x7f212019bff0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:36:59.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.482+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 >> 172.21.15.32:0/2050139404 conn(0x7f2120101610 msgr2=0x7f2120102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:36:59.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.482+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 shutdown_connections 2023-08-11T15:36:59.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:36:59.482+0000 7f2125602700 1 -- 172.21.15.32:0/2050139404 wait complete. 2023-08-11T15:36:59.877 INFO:teuthology.orchestra.run.smithi032.stdout:Restarting the monitor... 2023-08-11T15:37:00.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:36:59 smithi032 systemd[1]: Stopping Ceph mon.a for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:37:00.572 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:00 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-a[96572]: 2023-08-11T15:37:00.316+0000 7fcc330ee700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:37:00.572 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:00 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-a[96572]: 2023-08-11T15:37:00.316+0000 7fcc330ee700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-08-11T15:37:00.953 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:00 smithi032 bash[96934]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-a 2023-08-11T15:37:01.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:00 smithi032 systemd[1]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a.service: Succeeded. 2023-08-11T15:37:01.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:00 smithi032 systemd[1]: Stopped Ceph mon.a for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:37:01.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:00 smithi032 systemd[1]: Starting Ceph mon.a for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:37:01.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 podman[97063]: 2023-08-11T15:37:01.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 bash[97063]: 206d2b89c8f6f6c95452726aa8b353bf3dd8c76596b3225a9ca02b4db437bfbd 2023-08-11T15:37:01.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: set uid:gid to 167:167 (ceph:ceph) 2023-08-11T15:37:01.742 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev), process ceph-mon, pid 7 2023-08-11T15:37:01.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: pidfile_write: ignore empty --pid-file 2023-08-11T15:37:01.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: load: jerasure load: lrc 2023-08-11T15:37:01.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: RocksDB version: 7.9.2 2023-08-11T15:37:01.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Git sha 0 2023-08-11T15:37:01.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Compile date 2023-08-10 15:31:53 2023-08-11T15:37:01.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: DB SUMMARY 2023-08-11T15:37:01.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: DB Session ID: XZG27YYDAEMDWNNP7TC9 2023-08-11T15:37:01.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: CURRENT file: CURRENT 2023-08-11T15:37:01.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: IDENTITY file: IDENTITY 2023-08-11T15:37:01.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-08-11T15:37:01.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-08-11T15:37:01.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97132 ; 2023-08-11T15:37:01.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.error_if_exists: 0 2023-08-11T15:37:01.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.create_if_missing: 0 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.paranoid_checks: 1 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.env: 0x5561376ddda0 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.fs: PosixFileSystem 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.info_log: 0x5561387a3460 2023-08-11T15:37:01.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_file_opening_threads: 16 2023-08-11T15:37:01.746 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.statistics: (nil) 2023-08-11T15:37:01.746 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.use_fsync: 0 2023-08-11T15:37:01.746 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_log_file_size: 0 2023-08-11T15:37:01.746 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-11T15:37:01.746 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-11T15:37:01.746 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.keep_log_file_num: 1000 2023-08-11T15:37:01.746 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.recycle_log_file_num: 0 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.allow_fallocate: 1 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.allow_mmap_reads: 0 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.allow_mmap_writes: 0 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.use_direct_reads: 0 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.create_missing_column_families: 0 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.db_log_dir: 2023-08-11T15:37:01.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.wal_dir: 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.advise_random_on_open: 1 2023-08-11T15:37:01.748 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.db_write_buffer_size: 0 2023-08-11T15:37:01.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.write_buffer_manager: 0x55613875c3c0 2023-08-11T15:37:01.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-11T15:37:01.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-11T15:37:01.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-11T15:37:01.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.rate_limiter: (nil) 2023-08-11T15:37:01.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-11T15:37:01.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.wal_recovery_mode: 2 2023-08-11T15:37:01.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.enable_thread_tracking: 0 2023-08-11T15:37:01.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.enable_pipelined_write: 0 2023-08-11T15:37:01.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.unordered_write: 0 2023-08-11T15:37:01.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-11T15:37:01.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-11T15:37:01.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-11T15:37:01.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-11T15:37:01.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.row_cache: None 2023-08-11T15:37:01.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.wal_filter: None 2023-08-11T15:37:01.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-11T15:37:01.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.allow_ingest_behind: 0 2023-08-11T15:37:01.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.two_write_queues: 0 2023-08-11T15:37:01.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.manual_wal_flush: 0 2023-08-11T15:37:01.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.wal_compression: 0 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.atomic_flush: 0 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.log_readahead_size: 0 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.best_efforts_recovery: 0 2023-08-11T15:37:01.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-11T15:37:01.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-11T15:37:01.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.allow_data_in_errors: 0 2023-08-11T15:37:01.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.db_host_id: __hostname__ 2023-08-11T15:37:01.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-11T15:37:01.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_background_jobs: 2 2023-08-11T15:37:01.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_background_compactions: -1 2023-08-11T15:37:01.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_subcompactions: 1 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_total_wal_size: 0 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-11T15:37:01.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_open_files: -1 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bytes_per_sync: 0 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_readahead_size: 0 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_background_flushes: -1 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Compression algorithms supported: 2023-08-11T15:37:01.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-11T15:37:01.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kZSTD supported: 0 2023-08-11T15:37:01.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kXpressCompression supported: 0 2023-08-11T15:37:01.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kLZ4HCCompression supported: 1 2023-08-11T15:37:01.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kZlibCompression supported: 1 2023-08-11T15:37:01.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kSnappyCompression supported: 1 2023-08-11T15:37:01.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kLZ4Compression supported: 1 2023-08-11T15:37:01.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: kBZip2Compression supported: 0 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.merge_operator: 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_filter: None 2023-08-11T15:37:01.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_filter_factory: None 2023-08-11T15:37:01.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.sst_partitioner_factory: None 2023-08-11T15:37:01.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-11T15:37:01.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-11T15:37:01.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5561387a3560) 2023-08-11T15:37:01.758 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks: 1 2023-08-11T15:37:01.759 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-11T15:37:01.759 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-11T15:37:01.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_top_level_index_and_filter: 1 2023-08-11T15:37:01.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_type: 0 2023-08-11T15:37:01.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_index_type: 0 2023-08-11T15:37:01.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_shortening: 1 2023-08-11T15:37:01.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-11T15:37:01.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: checksum: 4 2023-08-11T15:37:01.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: no_block_cache: 0 2023-08-11T15:37:01.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache: 0x5561387fb1f0 2023-08-11T15:37:01.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_name: BinnedLRUCache 2023-08-11T15:37:01.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_options: 2023-08-11T15:37:01.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: capacity : 536870912 2023-08-11T15:37:01.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_shard_bits : 4 2023-08-11T15:37:01.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: strict_capacity_limit : 0 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: high_pri_pool_ratio: 0.000 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_compressed: (nil) 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: persistent_cache: (nil) 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size: 4096 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size_deviation: 10 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_restart_interval: 16 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_block_restart_interval: 1 2023-08-11T15:37:01.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: metadata_block_size: 4096 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: partition_filters: 0 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: use_delta_encoding: 1 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: filter_policy: bloomfilter 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: whole_key_filtering: 1 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: verify_compression: 0 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: read_amp_bytes_per_bit: 0 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: format_version: 5 2023-08-11T15:37:01.763 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_index_compression: 1 2023-08-11T15:37:01.764 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_align: 0 2023-08-11T15:37:01.764 INFO:journalctl@ceph.mon.a.smithi032.stdout: max_auto_readahead_size: 262144 2023-08-11T15:37:01.764 INFO:journalctl@ceph.mon.a.smithi032.stdout: prepopulate_block_cache: 0 2023-08-11T15:37:01.764 INFO:journalctl@ceph.mon.a.smithi032.stdout: initial_auto_readahead_size: 8192 2023-08-11T15:37:01.764 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2023-08-11T15:37:01.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.write_buffer_size: 33554432 2023-08-11T15:37:01.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_write_buffer_number: 2 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression: NoCompression 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression: Disabled 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.prefix_extractor: nullptr 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.num_levels: 7 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-11T15:37:01.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-11T15:37:01.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-11T15:37:01.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-11T15:37:01.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-11T15:37:01.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.level: 32767 2023-08-11T15:37:01.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.strategy: 0 2023-08-11T15:37:01.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-11T15:37:01.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-11T15:37:01.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.enabled: false 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.target_file_size_base: 67108864 2023-08-11T15:37:01.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-11T15:37:01.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-11T15:37:01.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-11T15:37:01.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-11T15:37:01.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-11T15:37:01.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-11T15:37:01.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-11T15:37:01.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-11T15:37:01.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-11T15:37:01.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-11T15:37:01.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-11T15:37:01.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.arena_block_size: 1048576 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.disable_auto_compactions: 0 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-11T15:37:01.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-11T15:37:01.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-11T15:37:01.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-11T15:37:01.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-11T15:37:01.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-11T15:37:01.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-11T15:37:01.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-11T15:37:01.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.table_properties_collectors: 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.inplace_update_support: 0 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-11T15:37:01.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.bloom_locality: 0 2023-08-11T15:37:01.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.max_successive_merges: 0 2023-08-11T15:37:01.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-11T15:37:01.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.paranoid_file_checks: 0 2023-08-11T15:37:01.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.force_consistency_checks: 1 2023-08-11T15:37:01.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.report_bg_io_stats: 0 2023-08-11T15:37:01.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.ttl: 2592000 2023-08-11T15:37:01.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.enable_blob_files: false 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.min_blob_size: 0 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.blob_file_size: 268435456 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-11T15:37:01.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-11T15:37:01.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-11T15:37:01.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-11T15:37:01.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.blob_file_starting_level: 0 2023-08-11T15:37:01.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-11T15:37:01.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-08-11T15:37:01.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-08-11T15:37:01.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3c29609a-3755-4625-b766-08d67c352017 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768221698347, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768221700347, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92452, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 311, "table_properties": {"data_size": 90452, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13482, "raw_average_key_size": 49, "raw_value_size": 83094, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 270, "num_filter_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1691768221, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3c29609a-3755-4625-b766-08d67c352017", "db_session_id": "XZG27YYDAEMDWNNP7TC9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768221700484, "job": 1, "event": "recovery_finished"} 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556138898000 2023-08-11T15:37:01.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: DB pointer 0x556138884000 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** DB Stats ** 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T15:37:01.778 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-11T15:37:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T15:37:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T15:37:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-08-11T15:37:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2023-08-11T15:37:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T15:37:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-11T15:37:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout: L0 2/0 92.12 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: Sum 2/0 92.12 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:37:01.780 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-08-11T15:37:01.781 INFO:journalctl@ceph.mon.a.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-11T15:37:01.781 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-08-11T15:37:01.781 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T15:37:01.781 INFO:journalctl@ceph.mon.a.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-11T15:37:01.781 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-11T15:37:01.781 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-11T15:37:01.781 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative compaction: 0.00 GB write, 13.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval compaction: 0.00 GB write, 13.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache BinnedLRUCache@0x5561387fb1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-08-11T15:37:01.782 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: starting mon.a rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???) e1 preinit fsid cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???).mds e1 new map 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???).mds e1 print_map 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout: e1 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T15:37:01.783 INFO:journalctl@ceph.mon.a.smithi032.stdout: legacy client fscid: -1 2023-08-11T15:37:01.784 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-08-11T15:37:01.784 INFO:journalctl@ceph.mon.a.smithi032.stdout: No filesystems configured 2023-08-11T15:37:01.784 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-11T15:37:01.784 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:37:01.784 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:37:01.784 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:37:01.784 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expand map: {default=false} 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta from 'false' to 'false' 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expanded map: {default=false} 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expand map: {default=info} 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta from 'info' to 'info' 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expanded map: {default=info} 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expand map: {default=daemon} 2023-08-11T15:37:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-11T15:37:01.786 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expanded map: {default=daemon} 2023-08-11T15:37:01.786 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expand map: {default=debug} 2023-08-11T15:37:01.786 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta from 'debug' to 'debug' 2023-08-11T15:37:01.786 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: expand_channel_meta expanded map: {default=debug} 2023-08-11T15:37:01.786 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 systemd[1]: Started Ceph mon.a for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:37:01.786 INFO:teuthology.orchestra.run.smithi032.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-08-11T15:37:02.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-11T15:37:02.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: monmap e1: 1 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} 2023-08-11T15:37:02.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: fsmap 2023-08-11T15:37:02.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: osdmap e1: 0 total, 0 up, 0 in 2023-08-11T15:37:02.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:01 smithi032 ceph-mon[97103]: mgrmap e1: no daemons active 2023-08-11T15:37:04.241 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.666+0000 7fb14284d700 1 Processor -- start 2023-08-11T15:37:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.666+0000 7fb14284d700 1 -- start start 2023-08-11T15:37:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.666+0000 7fb14284d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.666+0000 7fb14284d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c07a890 con 0x7fb13c07be60 2023-08-11T15:37:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.667+0000 7fb14184b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.667+0000 7fb14184b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35250/0 (socket says 172.21.15.32:35250) 2023-08-11T15:37:04.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.667+0000 7fb14184b700 1 -- 172.21.15.32:0/1909548735 learned_addr learned my addr 172.21.15.32:0/1909548735 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:04.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.668+0000 7fb14184b700 1 -- 172.21.15.32:0/1909548735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c07a9d0 con 0x7fb13c07be60 2023-08-11T15:37:04.244 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.668+0000 7fb14184b700 1 --2- 172.21.15.32:0/1909548735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c07a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1380098a0 tx=0x7fb138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08ad07c30d30dd5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:04.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.668+0000 7fb140849700 1 -- 172.21.15.32:0/1909548735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb138004160 con 0x7fb13c07be60 2023-08-11T15:37:04.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.668+0000 7fb140849700 1 -- 172.21.15.32:0/1909548735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb1380042c0 con 0x7fb13c07be60 2023-08-11T15:37:04.245 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.668+0000 7fb140849700 1 -- 172.21.15.32:0/1909548735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb138011a00 con 0x7fb13c07be60 2023-08-11T15:37:04.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.669+0000 7fb14284d700 1 -- 172.21.15.32:0/1909548735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 msgr2=0x7fb13c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:04.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.669+0000 7fb14284d700 1 --2- 172.21.15.32:0/1909548735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c07a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1380098a0 tx=0x7fb138009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:04.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.669+0000 7fb14284d700 1 -- 172.21.15.32:0/1909548735 shutdown_connections 2023-08-11T15:37:04.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.669+0000 7fb14284d700 1 --2- 172.21.15.32:0/1909548735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c07a2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:04.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.669+0000 7fb14284d700 1 -- 172.21.15.32:0/1909548735 >> 172.21.15.32:0/1909548735 conn(0x7fb13c101440 msgr2=0x7fb13c103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:04.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.669+0000 7fb14284d700 1 -- 172.21.15.32:0/1909548735 shutdown_connections 2023-08-11T15:37:04.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.669+0000 7fb14284d700 1 -- 172.21.15.32:0/1909548735 wait complete. 2023-08-11T15:37:04.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.670+0000 7fb14284d700 1 Processor -- start 2023-08-11T15:37:04.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.670+0000 7fb14284d700 1 -- start start 2023-08-11T15:37:04.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.670+0000 7fb14284d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:04.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.670+0000 7fb14284d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c19c360 con 0x7fb13c07be60 2023-08-11T15:37:04.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.671+0000 7fb14184b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:04.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.671+0000 7fb14184b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c19be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35258/0 (socket says 172.21.15.32:35258) 2023-08-11T15:37:04.250 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.671+0000 7fb14184b700 1 -- 172.21.15.32:0/3914246985 learned_addr learned my addr 172.21.15.32:0/3914246985 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:04.250 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.671+0000 7fb14184b700 1 -- 172.21.15.32:0/3914246985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138009550 con 0x7fb13c07be60 2023-08-11T15:37:04.250 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.671+0000 7fb14184b700 1 --2- 172.21.15.32:0/3914246985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c19be20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb138009130 tx=0x7fb138011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:04.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.672+0000 7fb132ffd700 1 -- 172.21.15.32:0/3914246985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb138003b70 con 0x7fb13c07be60 2023-08-11T15:37:04.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.672+0000 7fb132ffd700 1 -- 172.21.15.32:0/3914246985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb138003cd0 con 0x7fb13c07be60 2023-08-11T15:37:04.251 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.672+0000 7fb132ffd700 1 -- 172.21.15.32:0/3914246985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1380197d0 con 0x7fb13c07be60 2023-08-11T15:37:04.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.672+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c19c560 con 0x7fb13c07be60 2023-08-11T15:37:04.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.672+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c19c980 con 0x7fb13c07be60 2023-08-11T15:37:04.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.672+0000 7fb132ffd700 1 -- 172.21.15.32:0/3914246985 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb1380199c0 con 0x7fb13c07be60 2023-08-11T15:37:04.252 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.672+0000 7fb132ffd700 1 -- 172.21.15.32:0/3914246985 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb138025d50 con 0x7fb13c07be60 2023-08-11T15:37:04.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.673+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c195e70 con 0x7fb13c07be60 2023-08-11T15:37:04.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.676+0000 7fb132ffd700 1 -- 172.21.15.32:0/3914246985 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb13802a060 con 0x7fb13c07be60 2023-08-11T15:37:04.253 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.749+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fb13c10c500 con 0x7fb13c07be60 2023-08-11T15:37:04.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.752+0000 7fb132ffd700 1 -- 172.21.15.32:0/3914246985 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb138019e00 con 0x7fb13c07be60 2023-08-11T15:37:04.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.753+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 msgr2=0x7fb13c19be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:04.254 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.753+0000 7fb14284d700 1 --2- 172.21.15.32:0/3914246985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c19be20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb138009130 tx=0x7fb138011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:04.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.753+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 shutdown_connections 2023-08-11T15:37:04.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.753+0000 7fb14284d700 1 --2- 172.21.15.32:0/3914246985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13c07be60 0x7fb13c19be20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:04.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.753+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 >> 172.21.15.32:0/3914246985 conn(0x7fb13c101440 msgr2=0x7fb13c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:04.255 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.753+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 shutdown_connections 2023-08-11T15:37:04.256 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:03.753+0000 7fb14284d700 1 -- 172.21.15.32:0/3914246985 wait complete. 2023-08-11T15:37:04.256 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote config to /etc/ceph/ceph.conf 2023-08-11T15:37:04.267 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-08-11T15:37:04.268 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mgr... 2023-08-11T15:37:04.269 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 9283 ... 2023-08-11T15:37:04.270 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 8765 ... 2023-08-11T15:37:04.640 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.x 2023-08-11T15:37:04.640 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.x.service: Unit ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.x.service not loaded. 2023-08-11T15:37:04.856 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-cd36a838-385c-11ee-9b37-001a4aab830c.target.wants/ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-cd36a838-385c-11ee-9b37-001a4aab830c@.service. 2023-08-11T15:37:05.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:04 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3914246985' entity='client.admin' 2023-08-11T15:37:05.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:04 smithi032 systemd[1]: Starting Ceph mgr.x for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:37:05.605 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:05 smithi032 podman[97337]: 2023-08-11T15:37:05.675 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T15:37:05.675 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-08-11T15:37:05.692 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T15:37:05.692 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-08-11T15:37:05.693 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-08-11T15:37:05.693 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2023-08-11T15:37:05.710 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T15:37:05.710 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-08-11T15:37:05.727 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T15:37:05.727 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-08-11T15:37:05.727 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-08-11T15:37:05.727 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-08-11T15:37:05.727 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr to start... 2023-08-11T15:37:05.728 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr... 2023-08-11T15:37:05.908 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:05 smithi032 bash[97337]: 140588cec60c3d8f403017cd2217f6d85404e2293fcd5d558747900635278010 2023-08-11T15:37:05.908 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:05 smithi032 systemd[1]: Started Ceph mgr.x for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:37:05.909 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:05 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:05.795+0000 7fc86c91b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:37:05.909 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:05 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:05.905+0000 7fc86c91b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:37:06.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:06 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:06.027+0000 7fc86c91b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:37:06.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:06 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:06.155+0000 7fc86c91b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:37:06.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:06 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/428758420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-11T15:37:06.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:06 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:06.406+0000 7fc86c91b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:37:06.917 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:37:06.917 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-08-11T15:37:06.918 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "cd36a838-385c-11ee-9b37-001a4aab830c", 2023-08-11T15:37:06.918 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2023-08-11T15:37:06.918 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-11T15:37:06.918 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-11T15:37:06.918 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-11T15:37:06.918 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:06.918 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-11T15:37:06.919 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-11T15:37:06.919 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2023-08-11T15:37:06.919 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-08-11T15:37:06.919 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-11T15:37:06.919 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2023-08-11T15:37:06.919 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-08-11T15:37:06.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2023-08-11T15:37:06.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-11T15:37:06.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:06.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-11T15:37:06.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-11T15:37:06.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:06.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-11T15:37:06.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:06.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-11T15:37:06.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-11T15:37:06.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-11T15:37:06.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-11T15:37:06.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-11T15:37:06.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-11T15:37:06.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:06.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-11T15:37:06.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-11T15:37:06.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:06.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-11T15:37:06.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:06.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-11T15:37:06.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-11T15:37:06.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:06.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-08-11T15:37:06.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T15:37:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-11T15:37:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2023-08-11T15:36:56.367635+0000", 2023-08-11T15:37:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T15:37:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-11T15:37:06.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-08-11T15:37:06.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.309+0000 7f11b9961700 1 Processor -- start 2023-08-11T15:37:06.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.310+0000 7f11b9961700 1 -- start start 2023-08-11T15:37:06.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.310+0000 7f11b9961700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:06.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.310+0000 7f11b9961700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b407a8d0 con 0x7f11b407bea0 2023-08-11T15:37:06.927 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.311+0000 7f11b2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:06.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.311+0000 7f11b2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56186/0 (socket says 172.21.15.32:56186) 2023-08-11T15:37:06.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.311+0000 7f11b2ffd700 1 -- 172.21.15.32:0/3636549882 learned_addr learned my addr 172.21.15.32:0/3636549882 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:06.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.311+0000 7f11b2ffd700 1 -- 172.21.15.32:0/3636549882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b407aa10 con 0x7f11b407bea0 2023-08-11T15:37:06.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.312+0000 7f11b2ffd700 1 --2- 172.21.15.32:0/3636549882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b407a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f11a40098a0 tx=0x7f11a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d4bb5f3f6375fc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:06.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.312+0000 7f11b1ffb700 1 -- 172.21.15.32:0/3636549882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11a4004160 con 0x7f11b407bea0 2023-08-11T15:37:06.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.312+0000 7f11b1ffb700 1 -- 172.21.15.32:0/3636549882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11a40042c0 con 0x7f11b407bea0 2023-08-11T15:37:06.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.312+0000 7f11b1ffb700 1 -- 172.21.15.32:0/3636549882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11a4011a00 con 0x7f11b407bea0 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.313+0000 7f11b9961700 1 -- 172.21.15.32:0/3636549882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 msgr2=0x7f11b407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.313+0000 7f11b9961700 1 --2- 172.21.15.32:0/3636549882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b407a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f11a40098a0 tx=0x7f11a4009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.313+0000 7f11b9961700 1 -- 172.21.15.32:0/3636549882 shutdown_connections 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.313+0000 7f11b9961700 1 --2- 172.21.15.32:0/3636549882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b407a300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.313+0000 7f11b9961700 1 -- 172.21.15.32:0/3636549882 >> 172.21.15.32:0/3636549882 conn(0x7f11b4101610 msgr2=0x7f11b4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.313+0000 7f11b9961700 1 -- 172.21.15.32:0/3636549882 shutdown_connections 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.313+0000 7f11b9961700 1 -- 172.21.15.32:0/3636549882 wait complete. 2023-08-11T15:37:06.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.314+0000 7f11b9961700 1 Processor -- start 2023-08-11T15:37:06.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.315+0000 7f11b9961700 1 -- start start 2023-08-11T15:37:06.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.315+0000 7f11b9961700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:06.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.315+0000 7f11b9961700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b419c570 con 0x7f11b407bea0 2023-08-11T15:37:06.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.315+0000 7f11b2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:06.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.315+0000 7f11b2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56194/0 (socket says 172.21.15.32:56194) 2023-08-11T15:37:06.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.316+0000 7f11b2ffd700 1 -- 172.21.15.32:0/428758420 learned_addr learned my addr 172.21.15.32:0/428758420 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:06.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.316+0000 7f11b2ffd700 1 -- 172.21.15.32:0/428758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4009550 con 0x7f11b407bea0 2023-08-11T15:37:06.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.316+0000 7f11b2ffd700 1 --2- 172.21.15.32:0/428758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b419c030 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009130 tx=0x7f11a4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:06.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.316+0000 7f11b895f700 1 -- 172.21.15.32:0/428758420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11a4003ab0 con 0x7f11b407bea0 2023-08-11T15:37:06.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.316+0000 7f11b895f700 1 -- 172.21.15.32:0/428758420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11a4003c10 con 0x7f11b407bea0 2023-08-11T15:37:06.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.317+0000 7f11b895f700 1 -- 172.21.15.32:0/428758420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11a40247d0 con 0x7f11b407bea0 2023-08-11T15:37:06.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.317+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b419c770 con 0x7f11b407bea0 2023-08-11T15:37:06.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.317+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b419cb90 con 0x7f11b407bea0 2023-08-11T15:37:06.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.317+0000 7f11b895f700 1 -- 172.21.15.32:0/428758420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f11a4024d00 con 0x7f11b407bea0 2023-08-11T15:37:06.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.317+0000 7f11b895f700 1 -- 172.21.15.32:0/428758420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f11a4019450 con 0x7f11b407bea0 2023-08-11T15:37:06.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.318+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b4196050 con 0x7f11b407bea0 2023-08-11T15:37:06.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.321+0000 7f11b895f700 1 -- 172.21.15.32:0/428758420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f11a401abb0 con 0x7f11b407bea0 2023-08-11T15:37:06.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.411+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f11b410c440 con 0x7f11b407bea0 2023-08-11T15:37:06.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.411+0000 7f11b895f700 1 -- 172.21.15.32:0/428758420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f11a401ad50 con 0x7f11b407bea0 2023-08-11T15:37:06.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.412+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 msgr2=0x7f11b419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:06.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.412+0000 7f11b9961700 1 --2- 172.21.15.32:0/428758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b419c030 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009130 tx=0x7f11a4011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:06.934 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.412+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 shutdown_connections 2023-08-11T15:37:06.934 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.412+0000 7f11b9961700 1 --2- 172.21.15.32:0/428758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b407bea0 0x7f11b419c030 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:06.934 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.412+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 >> 172.21.15.32:0/428758420 conn(0x7f11b4101610 msgr2=0x7f11b41039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:06.934 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.413+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 shutdown_connections 2023-08-11T15:37:06.934 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:06.413+0000 7f11b9961700 1 -- 172.21.15.32:0/428758420 wait complete. 2023-08-11T15:37:06.935 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (1/15)... 2023-08-11T15:37:07.200 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:06 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:06.913+0000 7fc86c91b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:37:07.200 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:07 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:07.079+0000 7fc86c91b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:37:07.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:07 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:07.198+0000 7fc86c91b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:37:07.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:07 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:07.327+0000 7fc86c91b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:37:08.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:07 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:07.901+0000 7fc86c91b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:37:09.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:08 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:08.826+0000 7fc86c91b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:37:09.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:08 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:08.938+0000 7fc86c91b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:37:10.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:09 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:09.941+0000 7fc86c91b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:37:10.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:10 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:10.065+0000 7fc86c91b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:37:10.953 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:10 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:10.297+0000 7fc86c91b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:37:10.954 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:10 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:10.672+0000 7fc86c91b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:37:11.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:10 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:10.952+0000 7fc86c91b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:37:11.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:11 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:11.154+0000 7fc86c91b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:37:11.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:11 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:11.511+0000 7fc86c91b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:37:12.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:12 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:12.473+0000 7fc86c91b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:37:13.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:12 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:12.653+0000 7fc86c91b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:37:13.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:12 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:12.911+0000 7fc86c91b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:37:13.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:13 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:13.063+0000 7fc86c91b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:37:13.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:13 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:13.349+0000 7fc86c91b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:37:14.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:14.455+0000 7fc86c91b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:37:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: Activating manager daemon x 2023-08-11T15:37:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: mgrmap e2: x(active, starting, since 0.00423498s) 2023-08-11T15:37:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T15:37:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T15:37:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T15:37:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:37:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T15:37:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: Manager daemon x is now available 2023-08-11T15:37:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T15:37:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' 2023-08-11T15:37:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T15:37:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' 2023-08-11T15:37:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:14 smithi032 ceph-mon[97103]: from='mgr.14100 172.21.15.32:0/2089772888' entity='mgr.x' 2023-08-11T15:37:16.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:37:16.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-08-11T15:37:16.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "cd36a838-385c-11ee-9b37-001a4aab830c", 2023-08-11T15:37:16.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2023-08-11T15:37:16.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-11T15:37:16.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-11T15:37:16.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-11T15:37:16.417 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-08-11T15:37:16.418 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-08-11T15:37:16.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-11T15:37:16.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:16.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-11T15:37:16.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-11T15:37:16.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:16.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-11T15:37:16.419 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:16.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-11T15:37:16.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-11T15:37:16.420 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-11T15:37:16.426 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-11T15:37:16.426 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-11T15:37:16.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-11T15:37:16.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:16.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-11T15:37:16.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-11T15:37:16.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-11T15:37:16.427 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-11T15:37:16.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-11T15:37:16.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-11T15:37:16.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-11T15:37:16.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-11T15:37:16.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-11T15:37:16.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:16.428 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-11T15:37:16.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:16.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-11T15:37:16.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-11T15:37:16.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:16.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-11T15:37:16.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-11T15:37:16.429 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-11T15:37:16.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-11T15:37:16.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-11T15:37:16.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-11T15:37:16.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2023-08-11T15:37:16.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-08-11T15:37:16.430 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T15:37:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-11T15:37:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T15:37:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2023-08-11T15:36:56.367635+0000", 2023-08-11T15:37:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T15:37:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-08-11T15:37:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-11T15:37:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-08-11T15:37:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.649+0000 7fbe2a0ba700 1 Processor -- start 2023-08-11T15:37:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.649+0000 7fbe2a0ba700 1 -- start start 2023-08-11T15:37:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.649+0000 7fbe2a0ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe24106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.649+0000 7fbe2a0ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24106a60 con 0x7fbe24106070 2023-08-11T15:37:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.650+0000 7fbe237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe24106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:16.433 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.650+0000 7fbe237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe24106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38994/0 (socket says 172.21.15.32:38994) 2023-08-11T15:37:16.433 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.650+0000 7fbe237fe700 1 -- 172.21.15.32:0/4097268316 learned_addr learned my addr 172.21.15.32:0/4097268316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:16.433 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.650+0000 7fbe237fe700 1 -- 172.21.15.32:0/4097268316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe24107270 con 0x7fbe24106070 2023-08-11T15:37:16.433 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.650+0000 7fbe237fe700 1 --2- 172.21.15.32:0/4097268316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe24106490 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe140098a0 tx=0x7fbe14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61d447d1e848b20 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:16.433 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.650+0000 7fbe227fc700 1 -- 172.21.15.32:0/4097268316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe14004160 con 0x7fbe24106070 2023-08-11T15:37:16.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe227fc700 1 -- 172.21.15.32:0/4097268316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe140042c0 con 0x7fbe24106070 2023-08-11T15:37:16.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe227fc700 1 -- 172.21.15.32:0/4097268316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe14011a00 con 0x7fbe24106070 2023-08-11T15:37:16.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/4097268316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 msgr2=0x7fbe24106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:16.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe2a0ba700 1 --2- 172.21.15.32:0/4097268316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe24106490 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe140098a0 tx=0x7fbe14009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:16.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/4097268316 shutdown_connections 2023-08-11T15:37:16.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe2a0ba700 1 --2- 172.21.15.32:0/4097268316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe24106490 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:16.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/4097268316 >> 172.21.15.32:0/4097268316 conn(0x7fbe24101610 msgr2=0x7fbe24103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:16.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/4097268316 shutdown_connections 2023-08-11T15:37:16.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.651+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/4097268316 wait complete. 2023-08-11T15:37:16.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.653+0000 7fbe2a0ba700 1 Processor -- start 2023-08-11T15:37:16.436 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.653+0000 7fbe2a0ba700 1 -- start start 2023-08-11T15:37:16.436 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.653+0000 7fbe2a0ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe2407bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:16.437 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.653+0000 7fbe2a0ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2407c190 con 0x7fbe24106070 2023-08-11T15:37:16.437 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.654+0000 7fbe237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe2407bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:16.438 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.654+0000 7fbe237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe2407bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39006/0 (socket says 172.21.15.32:39006) 2023-08-11T15:37:16.438 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.654+0000 7fbe237fe700 1 -- 172.21.15.32:0/1640388143 learned_addr learned my addr 172.21.15.32:0/1640388143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:16.438 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.654+0000 7fbe237fe700 1 -- 172.21.15.32:0/1640388143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14009550 con 0x7fbe24106070 2023-08-11T15:37:16.439 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.654+0000 7fbe237fe700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe2407bc50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009fb0 tx=0x7fbe14011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:16.439 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.655+0000 7fbe20ff9700 1 -- 172.21.15.32:0/1640388143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe14003a40 con 0x7fbe24106070 2023-08-11T15:37:16.439 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.655+0000 7fbe20ff9700 1 -- 172.21.15.32:0/1640388143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe14003ba0 con 0x7fbe24106070 2023-08-11T15:37:16.440 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.655+0000 7fbe20ff9700 1 -- 172.21.15.32:0/1640388143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe1401a750 con 0x7fbe24106070 2023-08-11T15:37:16.440 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.655+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2407a2a0 con 0x7fbe24106070 2023-08-11T15:37:16.440 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.655+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2407a6c0 con 0x7fbe24106070 2023-08-11T15:37:16.441 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.656+0000 7fbe20ff9700 1 -- 172.21.15.32:0/1640388143 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 3) v1 ==== 45018+0+0 (secure 0 0 0) 0x7fbe14024b60 con 0x7fbe24106070 2023-08-11T15:37:16.441 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.656+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe24061f50 con 0x7fbe24106070 2023-08-11T15:37:16.441 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.656+0000 7fbe20ff9700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fbe0c037fa0 0x7fbe0c03a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:16.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.656+0000 7fbe20ff9700 1 -- 172.21.15.32:0/1640388143 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbe1404b940 con 0x7fbe24106070 2023-08-11T15:37:16.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.657+0000 7fbe22ffd700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fbe0c037fa0 0x7fbe0c03a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:16.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.657+0000 7fbe22ffd700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fbe0c037fa0 0x7fbe0c03a460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe18006fd0 tx=0x7fbe18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:16.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.662+0000 7fbe20ff9700 1 -- 172.21.15.32:0/1640388143 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbe14024430 con 0x7fbe24106070 2023-08-11T15:37:16.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.953+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbe2407b0d0 con 0x7fbe24106070 2023-08-11T15:37:16.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.953+0000 7fbe20ff9700 1 -- 172.21.15.32:0/1640388143 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbe140187c0 con 0x7fbe24106070 2023-08-11T15:37:16.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fbe0c037fa0 msgr2=0x7fbe0c03a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:16.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fbe0c037fa0 0x7fbe0c03a460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe18006fd0 tx=0x7fbe18006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:16.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 msgr2=0x7fbe2407bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:16.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe2407bc50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009fb0 tx=0x7fbe14011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:16.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 shutdown_connections 2023-08-11T15:37:16.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fbe0c037fa0 0x7fbe0c03a460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:16.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 --2- 172.21.15.32:0/1640388143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe24106070 0x7fbe2407bc50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:16.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 >> 172.21.15.32:0/1640388143 conn(0x7fbe24101610 msgr2=0x7fbe241039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:16.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 shutdown_connections 2023-08-11T15:37:16.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:15.956+0000 7fbe2a0ba700 1 -- 172.21.15.32:0/1640388143 wait complete. 2023-08-11T15:37:16.444 INFO:teuthology.orchestra.run.smithi032.stdout:mgr is available 2023-08-11T15:37:16.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:16 smithi032 ceph-mon[97103]: mgrmap e3: x(active, since 1.00905s) 2023-08-11T15:37:16.662 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:16 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1640388143' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-11T15:37:17.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:37:17.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2023-08-11T15:37:17.604 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:37:17.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-11T15:37:17.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-11T15:37:17.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-11T15:37:17.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-11T15:37:17.605 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:37:17.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-11T15:37:17.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-11T15:37:17.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-08-11T15:37:17.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2023-08-11T15:37:17.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-11T15:37:17.606 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-11T15:37:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-11T15:37:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-11T15:37:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-11T15:37:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.937+0000 7fb26180b700 1 Processor -- start 2023-08-11T15:37:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.937+0000 7fb26180b700 1 -- start start 2023-08-11T15:37:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.938+0000 7fb26180b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.938+0000 7fb26180b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c07a8f0 con 0x7fb25c07bec0 2023-08-11T15:37:17.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.938+0000 7fb25affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:17.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.938+0000 7fb25affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39018/0 (socket says 172.21.15.32:39018) 2023-08-11T15:37:17.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.938+0000 7fb25affd700 1 -- 172.21.15.32:0/3941029545 learned_addr learned my addr 172.21.15.32:0/3941029545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:17.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.939+0000 7fb25affd700 1 -- 172.21.15.32:0/3941029545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c07aa30 con 0x7fb25c07bec0 2023-08-11T15:37:17.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.939+0000 7fb25affd700 1 --2- 172.21.15.32:0/3941029545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c07a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0098a0 tx=0x7fb24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760c5c12f2b217d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:17.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.939+0000 7fb259ffb700 1 -- 172.21.15.32:0/3941029545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb24c004160 con 0x7fb25c07bec0 2023-08-11T15:37:17.608 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.939+0000 7fb259ffb700 1 -- 172.21.15.32:0/3941029545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb24c0042c0 con 0x7fb25c07bec0 2023-08-11T15:37:17.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.939+0000 7fb259ffb700 1 -- 172.21.15.32:0/3941029545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb24c011a00 con 0x7fb25c07bec0 2023-08-11T15:37:17.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.940+0000 7fb26180b700 1 -- 172.21.15.32:0/3941029545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 msgr2=0x7fb25c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:17.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.940+0000 7fb26180b700 1 --2- 172.21.15.32:0/3941029545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c07a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0098a0 tx=0x7fb24c009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:17.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.940+0000 7fb26180b700 1 -- 172.21.15.32:0/3941029545 shutdown_connections 2023-08-11T15:37:17.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.940+0000 7fb26180b700 1 --2- 172.21.15.32:0/3941029545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c07a320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:17.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.940+0000 7fb26180b700 1 -- 172.21.15.32:0/3941029545 >> 172.21.15.32:0/3941029545 conn(0x7fb25c101610 msgr2=0x7fb25c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:17.609 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.940+0000 7fb26180b700 1 -- 172.21.15.32:0/3941029545 shutdown_connections 2023-08-11T15:37:17.610 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.940+0000 7fb26180b700 1 -- 172.21.15.32:0/3941029545 wait complete. 2023-08-11T15:37:17.610 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.941+0000 7fb26180b700 1 Processor -- start 2023-08-11T15:37:17.610 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.941+0000 7fb26180b700 1 -- start start 2023-08-11T15:37:17.610 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.941+0000 7fb26180b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:17.610 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.941+0000 7fb26180b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c19c540 con 0x7fb25c07bec0 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb25affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb25affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c19c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39030/0 (socket says 172.21.15.32:39030) 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb25affd700 1 -- 172.21.15.32:0/230744794 learned_addr learned my addr 172.21.15.32:0/230744794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb25affd700 1 -- 172.21.15.32:0/230744794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c009550 con 0x7fb25c07bec0 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb25affd700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c19c000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009130 tx=0x7fb24c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb260809700 1 -- 172.21.15.32:0/230744794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb24c003ab0 con 0x7fb25c07bec0 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb260809700 1 -- 172.21.15.32:0/230744794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb24c003c10 con 0x7fb25c07bec0 2023-08-11T15:37:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.942+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c19c740 con 0x7fb25c07bec0 2023-08-11T15:37:17.612 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.943+0000 7fb260809700 1 -- 172.21.15.32:0/230744794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb24c0247d0 con 0x7fb25c07bec0 2023-08-11T15:37:17.612 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.943+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c19cb60 con 0x7fb25c07bec0 2023-08-11T15:37:17.612 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.944+0000 7fb260809700 1 -- 172.21.15.32:0/230744794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7fb24c01a430 con 0x7fb25c07bec0 2023-08-11T15:37:17.612 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.944+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c196050 con 0x7fb25c07bec0 2023-08-11T15:37:17.612 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.944+0000 7fb260809700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fb244038110 0x7fb24403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:17.612 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.944+0000 7fb260809700 1 -- 172.21.15.32:0/230744794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb24c04bc90 con 0x7fb25c07bec0 2023-08-11T15:37:17.612 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.944+0000 7fb25a7fc700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fb244038110 0x7fb24403a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:17.613 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.945+0000 7fb25a7fc700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fb244038110 0x7fb24403a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb250006fd0 tx=0x7fb250006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:17.613 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:16.949+0000 7fb260809700 1 -- 172.21.15.32:0/230744794 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb24c01ad90 con 0x7fb25c07bec0 2023-08-11T15:37:17.613 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.158+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb25c061fa0 con 0x7fb25c07bec0 2023-08-11T15:37:17.613 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.159+0000 7fb260809700 1 -- 172.21.15.32:0/230744794 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fb24c018350 con 0x7fb25c07bec0 2023-08-11T15:37:17.613 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fb244038110 msgr2=0x7fb24403a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:17.613 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fb244038110 0x7fb24403a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb250006fd0 tx=0x7fb250006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:17.613 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 msgr2=0x7fb25c19c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c19c000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009130 tx=0x7fb24c011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 shutdown_connections 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fb244038110 0x7fb24403a5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 --2- 172.21.15.32:0/230744794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c07bec0 0x7fb25c19c000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 >> 172.21.15.32:0/230744794 conn(0x7fb25c101610 msgr2=0x7fb25c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 shutdown_connections 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:17.162+0000 7fb26180b700 1 -- 172.21.15.32:0/230744794 wait complete. 2023-08-11T15:37:17.614 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling cephadm module... 2023-08-11T15:37:17.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:17 smithi032 ceph-mon[97103]: mgrmap e4: x(active, since 2s) 2023-08-11T15:37:17.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:17 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/230744794' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-11T15:37:18.741 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:18 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1017903850' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-08-11T15:37:18.742 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: ignoring --setuser ceph since I am not root 2023-08-11T15:37:18.743 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: ignoring --setgroup ceph since I am not root 2023-08-11T15:37:18.884 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.113+0000 7fa9237de700 1 Processor -- start 2023-08-11T15:37:18.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.113+0000 7fa9237de700 1 -- start start 2023-08-11T15:37:18.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.114+0000 7fa9237de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:18.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.114+0000 7fa9237de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c07a8d0 con 0x7fa91c07bea0 2023-08-11T15:37:18.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.114+0000 7fa92157a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:18.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.115+0000 7fa92157a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39042/0 (socket says 172.21.15.32:39042) 2023-08-11T15:37:18.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.115+0000 7fa92157a700 1 -- 172.21.15.32:0/397345767 learned_addr learned my addr 172.21.15.32:0/397345767 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:18.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.115+0000 7fa92157a700 1 -- 172.21.15.32:0/397345767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c07aa10 con 0x7fa91c07bea0 2023-08-11T15:37:18.891 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.115+0000 7fa92157a700 1 --2- 172.21.15.32:0/397345767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c07a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098a0 tx=0x7fa918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a90754379f226e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:18.891 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.115+0000 7fa913fff700 1 -- 172.21.15.32:0/397345767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa918004160 con 0x7fa91c07bea0 2023-08-11T15:37:18.891 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.115+0000 7fa913fff700 1 -- 172.21.15.32:0/397345767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa9180042c0 con 0x7fa91c07bea0 2023-08-11T15:37:18.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.115+0000 7fa913fff700 1 -- 172.21.15.32:0/397345767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa918011a00 con 0x7fa91c07bea0 2023-08-11T15:37:18.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.116+0000 7fa9237de700 1 -- 172.21.15.32:0/397345767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 msgr2=0x7fa91c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:18.892 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.116+0000 7fa9237de700 1 --2- 172.21.15.32:0/397345767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c07a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098a0 tx=0x7fa918009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:18.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.116+0000 7fa9237de700 1 -- 172.21.15.32:0/397345767 shutdown_connections 2023-08-11T15:37:18.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.116+0000 7fa9237de700 1 --2- 172.21.15.32:0/397345767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c07a300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:18.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.116+0000 7fa9237de700 1 -- 172.21.15.32:0/397345767 >> 172.21.15.32:0/397345767 conn(0x7fa91c101610 msgr2=0x7fa91c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:18.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.116+0000 7fa9237de700 1 -- 172.21.15.32:0/397345767 shutdown_connections 2023-08-11T15:37:18.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.116+0000 7fa9237de700 1 -- 172.21.15.32:0/397345767 wait complete. 2023-08-11T15:37:18.893 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.117+0000 7fa9237de700 1 Processor -- start 2023-08-11T15:37:18.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.117+0000 7fa9237de700 1 -- start start 2023-08-11T15:37:18.894 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.118+0000 7fa9237de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.118+0000 7fa9237de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c19c570 con 0x7fa91c07bea0 2023-08-11T15:37:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.118+0000 7fa92157a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.118+0000 7fa92157a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c19c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39056/0 (socket says 172.21.15.32:39056) 2023-08-11T15:37:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.118+0000 7fa92157a700 1 -- 172.21.15.32:0/1017903850 learned_addr learned my addr 172.21.15.32:0/1017903850 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.118+0000 7fa92157a700 1 -- 172.21.15.32:0/1017903850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918009550 con 0x7fa91c07bea0 2023-08-11T15:37:18.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.118+0000 7fa92157a700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c19c030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa918009130 tx=0x7fa918011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:18.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.119+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa918003ab0 con 0x7fa91c07bea0 2023-08-11T15:37:18.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.119+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa918003c10 con 0x7fa91c07bea0 2023-08-11T15:37:18.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.119+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9180247d0 con 0x7fa91c07bea0 2023-08-11T15:37:18.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.119+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c19c770 con 0x7fa91c07bea0 2023-08-11T15:37:18.896 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.119+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c19cb90 con 0x7fa91c07bea0 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.120+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7fa91801abb0 con 0x7fa91c07bea0 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.120+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c196050 con 0x7fa91c07bea0 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.120+0000 7fa9127fc700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fa908038110 0x7fa90803a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.120+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa91804bcb0 con 0x7fa91c07bea0 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.121+0000 7fa920d79700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fa908038110 0x7fa90803a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.121+0000 7fa920d79700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fa908038110 0x7fa90803a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa90c006fd0 tx=0x7fa90c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.125+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa918018710 con 0x7fa91c07bea0 2023-08-11T15:37:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.391+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa91c061fa0 con 0x7fa91c07bea0 2023-08-11T15:37:18.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.472+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fa91802d770 con 0x7fa91c07bea0 2023-08-11T15:37:18.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.473+0000 7fa9127fc700 1 -- 172.21.15.32:0/1017903850 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa91802ad60 con 0x7fa91c07bea0 2023-08-11T15:37:18.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fa908038110 msgr2=0x7fa90803a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:18.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fa908038110 0x7fa90803a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa90c006fd0 tx=0x7fa90c006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:18.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 msgr2=0x7fa91c19c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:18.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c19c030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa918009130 tx=0x7fa918011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:18.898 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 shutdown_connections 2023-08-11T15:37:18.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fa908038110 0x7fa90803a5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:18.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 --2- 172.21.15.32:0/1017903850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa91c07bea0 0x7fa91c19c030 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:18.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 >> 172.21.15.32:0/1017903850 conn(0x7fa91c101610 msgr2=0x7fa91c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:18.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 shutdown_connections 2023-08-11T15:37:18.899 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:18.476+0000 7fa9237de700 1 -- 172.21.15.32:0/1017903850 wait complete. 2023-08-11T15:37:19.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:18.740+0000 7fe0f12c8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:37:19.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:18.848+0000 7fe0f12c8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:37:19.054 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:18.974+0000 7fe0f12c8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:37:19.473 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:19 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:19.101+0000 7fe0f12c8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:37:19.474 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:19 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:19.348+0000 7fe0f12c8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:37:19.800 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:19 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1017903850' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-08-11T15:37:19.800 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:19 smithi032 ceph-mon[97103]: mgrmap e5: x(active, since 4s) 2023-08-11T15:37:20.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:19 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:19.878+0000 7fe0f12c8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:37:20.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:20.045+0000 7fe0f12c8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:37:20.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-08-11T15:37:20.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-08-11T15:37:20.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-11T15:37:20.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-11T15:37:20.173 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-11T15:37:20.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-08-11T15:37:20.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.436+0000 7f8b72387700 1 Processor -- start 2023-08-11T15:37:20.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.436+0000 7f8b72387700 1 -- start start 2023-08-11T15:37:20.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.436+0000 7f8b72387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:20.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.436+0000 7f8b72387700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c07a8b0 con 0x7f8b6c07be80 2023-08-11T15:37:20.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.437+0000 7f8b6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:20.174 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.437+0000 7f8b6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39072/0 (socket says 172.21.15.32:39072) 2023-08-11T15:37:20.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.437+0000 7f8b6bfff700 1 -- 172.21.15.32:0/1952723126 learned_addr learned my addr 172.21.15.32:0/1952723126 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:20.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.437+0000 7f8b6bfff700 1 -- 172.21.15.32:0/1952723126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c07a9f0 con 0x7f8b6c07be80 2023-08-11T15:37:20.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.437+0000 7f8b6bfff700 1 --2- 172.21.15.32:0/1952723126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c07a2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0098a0 tx=0x7f8b5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de628e2bb7411152 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:20.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.437+0000 7f8b6affd700 1 -- 172.21.15.32:0/1952723126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b5c004160 con 0x7f8b6c07be80 2023-08-11T15:37:20.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b6affd700 1 -- 172.21.15.32:0/1952723126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b5c0042c0 con 0x7f8b6c07be80 2023-08-11T15:37:20.175 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b6affd700 1 -- 172.21.15.32:0/1952723126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b5c011a00 con 0x7f8b6c07be80 2023-08-11T15:37:20.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b72387700 1 -- 172.21.15.32:0/1952723126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 msgr2=0x7f8b6c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:20.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b72387700 1 --2- 172.21.15.32:0/1952723126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c07a2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0098a0 tx=0x7f8b5c009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:20.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b72387700 1 -- 172.21.15.32:0/1952723126 shutdown_connections 2023-08-11T15:37:20.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b72387700 1 --2- 172.21.15.32:0/1952723126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c07a2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:20.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b72387700 1 -- 172.21.15.32:0/1952723126 >> 172.21.15.32:0/1952723126 conn(0x7f8b6c101610 msgr2=0x7f8b6c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:20.176 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b72387700 1 -- 172.21.15.32:0/1952723126 shutdown_connections 2023-08-11T15:37:20.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.438+0000 7f8b72387700 1 -- 172.21.15.32:0/1952723126 wait complete. 2023-08-11T15:37:20.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.440+0000 7f8b72387700 1 Processor -- start 2023-08-11T15:37:20.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.440+0000 7f8b72387700 1 -- start start 2023-08-11T15:37:20.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.441+0000 7f8b72387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:20.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.441+0000 7f8b72387700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c19c530 con 0x7f8b6c07be80 2023-08-11T15:37:20.177 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.441+0000 7f8b6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:20.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.441+0000 7f8b6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39082/0 (socket says 172.21.15.32:39082) 2023-08-11T15:37:20.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.441+0000 7f8b6bfff700 1 -- 172.21.15.32:0/3247223981 learned_addr learned my addr 172.21.15.32:0/3247223981 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:20.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.441+0000 7f8b6bfff700 1 -- 172.21.15.32:0/3247223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c009550 con 0x7f8b6c07be80 2023-08-11T15:37:20.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.442+0000 7f8b6bfff700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c19bff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009130 tx=0x7f8b5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:20.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.442+0000 7f8b697fa700 1 -- 172.21.15.32:0/3247223981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b5c003b70 con 0x7f8b6c07be80 2023-08-11T15:37:20.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.442+0000 7f8b697fa700 1 -- 172.21.15.32:0/3247223981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b5c003cd0 con 0x7f8b6c07be80 2023-08-11T15:37:20.178 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.442+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c19c730 con 0x7f8b6c07be80 2023-08-11T15:37:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.442+0000 7f8b697fa700 1 -- 172.21.15.32:0/3247223981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b5c0197d0 con 0x7f8b6c07be80 2023-08-11T15:37:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.442+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c19cb50 con 0x7f8b6c07be80 2023-08-11T15:37:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.443+0000 7f8b697fa700 1 -- 172.21.15.32:0/3247223981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f8b5c01abb0 con 0x7f8b6c07be80 2023-08-11T15:37:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.443+0000 7f8b697fa700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 0x7f8b5403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.443+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6c196040 con 0x7f8b6c07be80 2023-08-11T15:37:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.444+0000 7f8b697fa700 1 -- 172.21.15.32:0/3247223981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8b5c04c260 con 0x7f8b6c07be80 2023-08-11T15:37:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.444+0000 7f8b6b7fe700 1 -- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 msgr2=0x7f8b5403a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:20.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.444+0000 7f8b6b7fe700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 0x7f8b5403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:37:20.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.449+0000 7f8b697fa700 1 -- 172.21.15.32:0/3247223981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8b5c01a430 con 0x7f8b6c07be80 2023-08-11T15:37:20.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.645+0000 7f8b6b7fe700 1 -- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 msgr2=0x7f8b5403a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:20.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.645+0000 7f8b6b7fe700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 0x7f8b5403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T15:37:20.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.773+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8b6c061f50 con 0x7f8b6c07be80 2023-08-11T15:37:20.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.774+0000 7f8b697fa700 1 -- 172.21.15.32:0/3247223981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8b5c0172b0 con 0x7f8b6c07be80 2023-08-11T15:37:20.180 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.776+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 msgr2=0x7f8b5403a5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:37:20.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.776+0000 7f8b72387700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 0x7f8b5403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:20.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.776+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 msgr2=0x7f8b6c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:20.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.776+0000 7f8b72387700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c19bff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009130 tx=0x7f8b5c011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:20.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.777+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 shutdown_connections 2023-08-11T15:37:20.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.777+0000 7f8b72387700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7f8b54038110 0x7f8b5403a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:20.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.777+0000 7f8b72387700 1 --2- 172.21.15.32:0/3247223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c07be80 0x7f8b6c19bff0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:20.181 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.777+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 >> 172.21.15.32:0/3247223981 conn(0x7f8b6c101610 msgr2=0x7f8b6c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:20.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.777+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 shutdown_connections 2023-08-11T15:37:20.182 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:19.777+0000 7f8b72387700 1 -- 172.21.15.32:0/3247223981 wait complete. 2023-08-11T15:37:20.182 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2023-08-11T15:37:20.182 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 5... 2023-08-11T15:37:20.454 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:20.168+0000 7fe0f12c8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:37:20.454 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:20.299+0000 7fe0f12c8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:37:20.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:20 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3247223981' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-11T15:37:21.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:20.878+0000 7fe0f12c8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:37:22.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:21 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:21.755+0000 7fe0f12c8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:37:22.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:21 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:21.865+0000 7fe0f12c8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:37:23.208 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:22.851+0000 7fe0f12c8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:37:23.208 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:22.974+0000 7fe0f12c8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:37:23.556 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:23.205+0000 7fe0f12c8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:37:23.859 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:23.577+0000 7fe0f12c8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:37:24.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:23.857+0000 7fe0f12c8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:37:24.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:24.060+0000 7fe0f12c8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:37:24.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:24.415+0000 7fe0f12c8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:37:25.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:25.369+0000 7fe0f12c8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:37:25.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:25.548+0000 7fe0f12c8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:37:26.240 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:25.803+0000 7fe0f12c8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:37:26.240 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:25.953+0000 7fe0f12c8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:37:26.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:26.237+0000 7fe0f12c8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:37:27.716 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:27.335+0000 7fe0f12c8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:37:28.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: Active manager daemon x restarted 2023-08-11T15:37:28.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: Activating manager daemon x 2023-08-11T15:37:28.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: osdmap e2: 0 total, 0 up, 0 in 2023-08-11T15:37:28.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: mgrmap e6: x(active, starting, since 0.00369829s) 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: Manager daemon x is now available 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: Found migration_current of "None". Setting to last migration. 2023-08-11T15:37:28.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:28.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:28.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:28.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:28.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:28.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T15:37:28.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T15:37:28.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:28.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:27 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:28.770 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-08-11T15:37:28.771 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-08-11T15:37:28.771 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-11T15:37:28.771 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-08-11T15:37:28.771 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.730+0000 7fc84624f700 1 Processor -- start 2023-08-11T15:37:28.771 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.731+0000 7fc84624f700 1 -- start start 2023-08-11T15:37:28.774 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.731+0000 7fc84624f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:28.775 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.731+0000 7fc84624f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84007a8b0 con 0x7fc84007be80 2023-08-11T15:37:28.775 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.732+0000 7fc83f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:28.775 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.732+0000 7fc83f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39098/0 (socket says 172.21.15.32:39098) 2023-08-11T15:37:28.776 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.732+0000 7fc83f7fe700 1 -- 172.21.15.32:0/2318266492 learned_addr learned my addr 172.21.15.32:0/2318266492 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:28.776 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.732+0000 7fc83f7fe700 1 -- 172.21.15.32:0/2318266492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84007a9f0 con 0x7fc84007be80 2023-08-11T15:37:28.776 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.732+0000 7fc83f7fe700 1 --2- 172.21.15.32:0/2318266492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84007a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8300098a0 tx=0x7fc830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a8c410f71db3ae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.733+0000 7fc83e7fc700 1 -- 172.21.15.32:0/2318266492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc830004160 con 0x7fc84007be80 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.733+0000 7fc83e7fc700 1 -- 172.21.15.32:0/2318266492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8300042c0 con 0x7fc84007be80 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.733+0000 7fc83e7fc700 1 -- 172.21.15.32:0/2318266492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc830011a00 con 0x7fc84007be80 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.733+0000 7fc84624f700 1 -- 172.21.15.32:0/2318266492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 msgr2=0x7fc84007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.733+0000 7fc84624f700 1 --2- 172.21.15.32:0/2318266492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84007a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8300098a0 tx=0x7fc830009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.734+0000 7fc84624f700 1 -- 172.21.15.32:0/2318266492 shutdown_connections 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.734+0000 7fc84624f700 1 --2- 172.21.15.32:0/2318266492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84007a2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.734+0000 7fc84624f700 1 -- 172.21.15.32:0/2318266492 >> 172.21.15.32:0/2318266492 conn(0x7fc840101610 msgr2=0x7fc840103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:28.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.734+0000 7fc84624f700 1 -- 172.21.15.32:0/2318266492 shutdown_connections 2023-08-11T15:37:28.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.734+0000 7fc84624f700 1 -- 172.21.15.32:0/2318266492 wait complete. 2023-08-11T15:37:28.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.735+0000 7fc84624f700 1 Processor -- start 2023-08-11T15:37:28.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.735+0000 7fc84624f700 1 -- start start 2023-08-11T15:37:28.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.736+0000 7fc84624f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:28.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.736+0000 7fc84624f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84019c530 con 0x7fc84007be80 2023-08-11T15:37:28.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.736+0000 7fc83f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.736+0000 7fc83f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84019bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39104/0 (socket says 172.21.15.32:39104) 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.736+0000 7fc83f7fe700 1 -- 172.21.15.32:0/430287710 learned_addr learned my addr 172.21.15.32:0/430287710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.737+0000 7fc83f7fe700 1 -- 172.21.15.32:0/430287710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc830009550 con 0x7fc84007be80 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.737+0000 7fc83f7fe700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84019bff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc830009130 tx=0x7fc830011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.737+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc830003b70 con 0x7fc84007be80 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.737+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc830003cd0 con 0x7fc84007be80 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.737+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8300197d0 con 0x7fc84007be80 2023-08-11T15:37:28.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.737+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84019c730 con 0x7fc84007be80 2023-08-11T15:37:28.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.737+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84019cb50 con 0x7fc84007be80 2023-08-11T15:37:28.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.738+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fc83001a430 con 0x7fc84007be80 2023-08-11T15:37:28.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.739+0000 7fc83cff9700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:28.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.739+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 --> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc82803ad30 con 0x7fc828038160 2023-08-11T15:37:28.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.739+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc83004c140 con 0x7fc84007be80 2023-08-11T15:37:28.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.739+0000 7fc83effd700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 msgr2=0x7fc82803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:28.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.739+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:37:28.781 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.940+0000 7fc83effd700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 msgr2=0x7fc82803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:28.781 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:20.940+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T15:37:28.781 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:21.341+0000 7fc83effd700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 msgr2=0x7fc82803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:28.781 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:21.341+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-11T15:37:28.781 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:22.142+0000 7fc83effd700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 msgr2=0x7fc82803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:28.781 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:22.142+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-11T15:37:28.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:23.744+0000 7fc83effd700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 msgr2=0x7fc82803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:28.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:23.744+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-11T15:37:28.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:26.947+0000 7fc83effd700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 msgr2=0x7fc82803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/114266517 2023-08-11T15:37:28.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:26.947+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-11T15:37:28.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:27.339+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 6) v1 ==== 44902+0+0 (secure 0 0 0) 0x7fc830022d90 con 0x7fc84007be80 2023-08-11T15:37:28.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:27.339+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 msgr2=0x7fc82803a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:37:28.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:27.339+0000 7fc83cff9700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.341+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7fc830017d90 con 0x7fc84007be80 2023-08-11T15:37:28.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.341+0000 7fc83cff9700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fc82803ba80 0x7fc82803de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:28.787 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.341+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc82803ad30 con 0x7fc82803ba80 2023-08-11T15:37:28.787 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.342+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fc82803ba80 0x7fc82803de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:28.787 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.343+0000 7fc83effd700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fc82803ba80 0x7fc82803de70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc834003a10 tx=0x7fc8340092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:28.788 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.344+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fc82803ad30 con 0x7fc82803ba80 2023-08-11T15:37:28.788 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.351+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc84019d330 con 0x7fc82803ba80 2023-08-11T15:37:28.788 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc83cff9700 1 -- 172.21.15.32:0/430287710 <== mgr.14116 v2:172.21.15.32:6800/4253467068 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc84019d330 con 0x7fc82803ba80 2023-08-11T15:37:28.789 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fc82803ba80 msgr2=0x7fc82803de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:28.789 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fc82803ba80 0x7fc82803de70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc834003a10 tx=0x7fc8340092b0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.789 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 msgr2=0x7fc84019bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:28.789 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84019bff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc830009130 tx=0x7fc830011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.790 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 shutdown_connections 2023-08-11T15:37:28.790 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fc82803ba80 0x7fc82803de70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.790 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84007be80 0x7fc84019bff0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.791 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 --2- 172.21.15.32:0/430287710 >> [v2:172.21.15.32:6800/114266517,v1:172.21.15.32:6801/114266517] conn(0x7fc828038160 0x7fc82803a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:28.791 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 >> 172.21.15.32:0/430287710 conn(0x7fc840101610 msgr2=0x7fc840102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:28.791 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 shutdown_connections 2023-08-11T15:37:28.791 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:28.352+0000 7fc84624f700 1 -- 172.21.15.32:0/430287710 wait complete. 2023-08-11T15:37:28.792 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 5 is available 2023-08-11T15:37:28.792 INFO:teuthology.orchestra.run.smithi032.stdout:Setting orchestrator backend to cephadm... 2023-08-11T15:37:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:28] ENGINE Bus STARTING 2023-08-11T15:37:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:28] ENGINE Serving on http://172.21.15.32:8765 2023-08-11T15:37:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:28] ENGINE Serving on https://172.21.15.32:7150 2023-08-11T15:37:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:28] ENGINE Bus STARTED 2023-08-11T15:37:29.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:29.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: mgrmap e7: x(active, since 1.00661s) 2023-08-11T15:37:29.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-08-11T15:37:29.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:28 smithi032 ceph-mon[97103]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-08-11T15:37:29.961 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.323+0000 7fe6b1564700 1 Processor -- start 2023-08-11T15:37:29.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.323+0000 7fe6b1564700 1 -- start start 2023-08-11T15:37:29.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.323+0000 7fe6b1564700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac104d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:29.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.323+0000 7fe6b1564700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac079e30 con 0x7fe6ac104950 2023-08-11T15:37:29.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.323+0000 7fe6aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac104d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:29.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.323+0000 7fe6aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac104d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39914/0 (socket says 172.21.15.32:39914) 2023-08-11T15:37:29.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.323+0000 7fe6aaffd700 1 -- 172.21.15.32:0/3185502820 learned_addr learned my addr 172.21.15.32:0/3185502820 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:29.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.324+0000 7fe6aaffd700 1 -- 172.21.15.32:0/3185502820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac1052b0 con 0x7fe6ac104950 2023-08-11T15:37:29.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.324+0000 7fe6aaffd700 1 --2- 172.21.15.32:0/3185502820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac104d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0098a0 tx=0x7fe69c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f320438949e6adc5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:29.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.324+0000 7fe6a9ffb700 1 -- 172.21.15.32:0/3185502820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c004160 con 0x7fe6ac104950 2023-08-11T15:37:29.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.324+0000 7fe6a9ffb700 1 -- 172.21.15.32:0/3185502820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe69c0042c0 con 0x7fe6ac104950 2023-08-11T15:37:29.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.324+0000 7fe6a9ffb700 1 -- 172.21.15.32:0/3185502820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c011a00 con 0x7fe6ac104950 2023-08-11T15:37:29.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.325+0000 7fe6b1564700 1 -- 172.21.15.32:0/3185502820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 msgr2=0x7fe6ac104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:29.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.325+0000 7fe6b1564700 1 --2- 172.21.15.32:0/3185502820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac104d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0098a0 tx=0x7fe69c009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:29.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.325+0000 7fe6b1564700 1 -- 172.21.15.32:0/3185502820 shutdown_connections 2023-08-11T15:37:29.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.325+0000 7fe6b1564700 1 --2- 172.21.15.32:0/3185502820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac104d70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:29.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.325+0000 7fe6b1564700 1 -- 172.21.15.32:0/3185502820 >> 172.21.15.32:0/3185502820 conn(0x7fe6ac100510 msgr2=0x7fe6ac102950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:29.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.325+0000 7fe6b1564700 1 -- 172.21.15.32:0/3185502820 shutdown_connections 2023-08-11T15:37:29.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.325+0000 7fe6b1564700 1 -- 172.21.15.32:0/3185502820 wait complete. 2023-08-11T15:37:29.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.326+0000 7fe6b1564700 1 Processor -- start 2023-08-11T15:37:29.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.326+0000 7fe6b1564700 1 -- start start 2023-08-11T15:37:29.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.327+0000 7fe6b1564700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:29.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.327+0000 7fe6b1564700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19c4f0 con 0x7fe6ac104950 2023-08-11T15:37:29.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.327+0000 7fe6aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:29.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.327+0000 7fe6aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39922/0 (socket says 172.21.15.32:39922) 2023-08-11T15:37:29.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.327+0000 7fe6aaffd700 1 -- 172.21.15.32:0/2845860596 learned_addr learned my addr 172.21.15.32:0/2845860596 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:29.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.327+0000 7fe6aaffd700 1 -- 172.21.15.32:0/2845860596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c009550 con 0x7fe6ac104950 2023-08-11T15:37:29.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.327+0000 7fe6aaffd700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac19bfb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009130 tx=0x7fe69c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:29.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.328+0000 7fe693fff700 1 -- 172.21.15.32:0/2845860596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c003b70 con 0x7fe6ac104950 2023-08-11T15:37:29.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.328+0000 7fe693fff700 1 -- 172.21.15.32:0/2845860596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe69c003cd0 con 0x7fe6ac104950 2023-08-11T15:37:29.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.328+0000 7fe693fff700 1 -- 172.21.15.32:0/2845860596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe69c0197d0 con 0x7fe6ac104950 2023-08-11T15:37:29.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.328+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac19c6f0 con 0x7fe6ac104950 2023-08-11T15:37:29.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.328+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac19cb10 con 0x7fe6ac104950 2023-08-11T15:37:29.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.329+0000 7fe693fff700 1 -- 172.21.15.32:0/2845860596 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7fe69c01abb0 con 0x7fe6ac104950 2023-08-11T15:37:29.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.329+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac195800 con 0x7fe6ac104950 2023-08-11T15:37:29.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.329+0000 7fe693fff700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fe694038030 0x7fe69403a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:29.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.329+0000 7fe693fff700 1 -- 172.21.15.32:0/2845860596 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe69c04bfe0 con 0x7fe6ac104950 2023-08-11T15:37:29.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.330+0000 7fe6aa7fc700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fe694038030 0x7fe69403a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:29.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.330+0000 7fe6aa7fc700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fe694038030 0x7fe69403a4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:29.972 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.335+0000 7fe693fff700 1 -- 172.21.15.32:0/2845860596 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe69c01a430 con 0x7fe6ac104950 2023-08-11T15:37:29.972 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.558+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6ac02cce0 con 0x7fe694038030 2023-08-11T15:37:29.972 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.565+0000 7fe693fff700 1 -- 172.21.15.32:0/2845860596 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe6ac02cce0 con 0x7fe694038030 2023-08-11T15:37:29.973 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.567+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fe694038030 msgr2=0x7fe69403a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:29.973 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.567+0000 7fe6b1564700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fe694038030 0x7fe69403a4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:29.973 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.567+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 msgr2=0x7fe6ac19bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:29.973 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.567+0000 7fe6b1564700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac19bfb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009130 tx=0x7fe69c011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:29.973 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.568+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 shutdown_connections 2023-08-11T15:37:29.974 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.568+0000 7fe6b1564700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fe694038030 0x7fe69403a4f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:29.974 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.568+0000 7fe6b1564700 1 --2- 172.21.15.32:0/2845860596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac104950 0x7fe6ac19bfb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:29.974 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.568+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 >> 172.21.15.32:0/2845860596 conn(0x7fe6ac100510 msgr2=0x7fe6ac190510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:29.974 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.568+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 shutdown_connections 2023-08-11T15:37:29.974 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:29.568+0000 7fe6b1564700 1 -- 172.21.15.32:0/2845860596 wait complete. 2023-08-11T15:37:30.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:30 smithi032 ceph-mon[97103]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:30.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:30 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:30.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:30 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:30.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:30 smithi032 ceph-mon[97103]: mgrmap e8: x(active, since 2s) 2023-08-11T15:37:31.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout value unchanged 2023-08-11T15:37:31.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.462+0000 7f784a83b700 1 Processor -- start 2023-08-11T15:37:31.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.462+0000 7f784a83b700 1 -- start start 2023-08-11T15:37:31.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.462+0000 7f784a83b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:31.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.462+0000 7f784a83b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784407a8d0 con 0x7f784407bea0 2023-08-11T15:37:31.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.462+0000 7f7843fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:31.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.462+0000 7f7843fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39934/0 (socket says 172.21.15.32:39934) 2023-08-11T15:37:31.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.462+0000 7f7843fff700 1 -- 172.21.15.32:0/2639046871 learned_addr learned my addr 172.21.15.32:0/2639046871 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:31.141 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.463+0000 7f7843fff700 1 -- 172.21.15.32:0/2639046871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784407aa10 con 0x7f784407bea0 2023-08-11T15:37:31.141 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.463+0000 7f7843fff700 1 --2- 172.21.15.32:0/2639046871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784407a300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f78340098a0 tx=0x7f7834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220339fe7b9ca541 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:31.141 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.463+0000 7f7842ffd700 1 -- 172.21.15.32:0/2639046871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7834004160 con 0x7f784407bea0 2023-08-11T15:37:31.142 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.463+0000 7f7842ffd700 1 -- 172.21.15.32:0/2639046871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f78340042c0 con 0x7f784407bea0 2023-08-11T15:37:31.142 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.463+0000 7f7842ffd700 1 -- 172.21.15.32:0/2639046871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7834011a00 con 0x7f784407bea0 2023-08-11T15:37:31.142 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.464+0000 7f784a83b700 1 -- 172.21.15.32:0/2639046871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 msgr2=0x7f784407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:31.142 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.464+0000 7f784a83b700 1 --2- 172.21.15.32:0/2639046871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784407a300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f78340098a0 tx=0x7f7834009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:31.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.464+0000 7f784a83b700 1 -- 172.21.15.32:0/2639046871 shutdown_connections 2023-08-11T15:37:31.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.464+0000 7f784a83b700 1 --2- 172.21.15.32:0/2639046871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784407a300 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:31.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.464+0000 7f784a83b700 1 -- 172.21.15.32:0/2639046871 >> 172.21.15.32:0/2639046871 conn(0x7f7844101610 msgr2=0x7f7844103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:31.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.464+0000 7f784a83b700 1 -- 172.21.15.32:0/2639046871 shutdown_connections 2023-08-11T15:37:31.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.464+0000 7f784a83b700 1 -- 172.21.15.32:0/2639046871 wait complete. 2023-08-11T15:37:31.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.465+0000 7f784a83b700 1 Processor -- start 2023-08-11T15:37:31.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.465+0000 7f784a83b700 1 -- start start 2023-08-11T15:37:31.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.465+0000 7f784a83b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:31.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.465+0000 7f784a83b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784419c570 con 0x7f784407bea0 2023-08-11T15:37:31.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.466+0000 7f7843fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:31.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.466+0000 7f7843fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39938/0 (socket says 172.21.15.32:39938) 2023-08-11T15:37:31.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.466+0000 7f7843fff700 1 -- 172.21.15.32:0/4162018186 learned_addr learned my addr 172.21.15.32:0/4162018186 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:31.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.466+0000 7f7843fff700 1 -- 172.21.15.32:0/4162018186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7834009550 con 0x7f784407bea0 2023-08-11T15:37:31.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.466+0000 7f7843fff700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784419c030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7834009130 tx=0x7f7834011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:31.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.466+0000 7f78417fa700 1 -- 172.21.15.32:0/4162018186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7834003ab0 con 0x7f784407bea0 2023-08-11T15:37:31.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.467+0000 7f78417fa700 1 -- 172.21.15.32:0/4162018186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7834003c10 con 0x7f784407bea0 2023-08-11T15:37:31.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.467+0000 7f78417fa700 1 -- 172.21.15.32:0/4162018186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78340247d0 con 0x7f784407bea0 2023-08-11T15:37:31.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.467+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784419c770 con 0x7f784407bea0 2023-08-11T15:37:31.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.467+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784419cb90 con 0x7f784407bea0 2023-08-11T15:37:31.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.468+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7844196050 con 0x7f784407bea0 2023-08-11T15:37:31.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.468+0000 7f78417fa700 1 -- 172.21.15.32:0/4162018186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f783401abb0 con 0x7f784407bea0 2023-08-11T15:37:31.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.469+0000 7f78417fa700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f782c038160 0x7f782c03a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:31.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.469+0000 7f78417fa700 1 -- 172.21.15.32:0/4162018186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f783404c030 con 0x7f784407bea0 2023-08-11T15:37:31.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.469+0000 7f78437fe700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f782c038160 0x7f782c03a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:31.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.469+0000 7f78437fe700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f782c038160 0x7f782c03a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7838006fd0 tx=0x7f7838006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:31.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.473+0000 7f78417fa700 1 -- 172.21.15.32:0/4162018186 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f783407ec30 con 0x7f784407bea0 2023-08-11T15:37:31.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.691+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f784402cd20 con 0x7f782c038160 2023-08-11T15:37:31.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.692+0000 7f78417fa700 1 -- 172.21.15.32:0/4162018186 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f784402cd20 con 0x7f782c038160 2023-08-11T15:37:31.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f782c038160 msgr2=0x7f782c03a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:31.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f782c038160 0x7f782c03a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7838006fd0 tx=0x7f7838006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:31.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 msgr2=0x7f784419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:31.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784419c030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7834009130 tx=0x7f7834011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:31.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 shutdown_connections 2023-08-11T15:37:31.151 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f782c038160 0x7f782c03a620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:31.151 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 --2- 172.21.15.32:0/4162018186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f784407bea0 0x7f784419c030 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:31.151 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 >> 172.21.15.32:0/4162018186 conn(0x7f7844101610 msgr2=0x7f78441039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:31.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 shutdown_connections 2023-08-11T15:37:31.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:30.695+0000 7f784a83b700 1 -- 172.21.15.32:0/4162018186 wait complete. 2023-08-11T15:37:31.152 INFO:teuthology.orchestra.run.smithi032.stdout:Generating ssh key... 2023-08-11T15:37:31.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:31 smithi032 ceph-mon[97103]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:32.804 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: Generating public/private rsa key pair. 2023-08-11T15:37:32.804 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: Your identification has been saved in /tmp/tmpn48ts_7q/key. 2023-08-11T15:37:32.804 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: Your public key has been saved in /tmp/tmpn48ts_7q/key.pub. 2023-08-11T15:37:32.804 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: The key fingerprint is: 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: SHA256:46e3YzXzNYbzLNqdfmtVnWirD8MBoKi/5+Us7oDHu0I ceph-cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: The key's randomart image is: 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: +---[RSA 3072]----+ 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | . | 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | . . . | 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | . . . . o| 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | . . o .o| 2023-08-11T15:37:32.805 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | . S .. o .| 2023-08-11T15:37:32.806 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | E+ . .. =+ oo| 2023-08-11T15:37:32.806 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: |.. = .. .=.+=.o| 2023-08-11T15:37:32.806 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | .. +o+ o+.+.o+o| 2023-08-11T15:37:32.806 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: | .+*+.o.o.+oo+=o| 2023-08-11T15:37:32.806 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: +----[SHA256]-----+ 2023-08-11T15:37:32.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.643+0000 7f1f899fe700 1 Processor -- start 2023-08-11T15:37:32.820 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.643+0000 7f1f899fe700 1 -- start start 2023-08-11T15:37:32.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.643+0000 7f1f899fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:32.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.643+0000 7f1f899fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8407a8b0 con 0x7f1f8407be80 2023-08-11T15:37:32.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.643+0000 7f1f889fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:32.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.643+0000 7f1f889fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39950/0 (socket says 172.21.15.32:39950) 2023-08-11T15:37:32.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.643+0000 7f1f889fc700 1 -- 172.21.15.32:0/560812998 learned_addr learned my addr 172.21.15.32:0/560812998 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:32.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.644+0000 7f1f889fc700 1 -- 172.21.15.32:0/560812998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8407a9f0 con 0x7f1f8407be80 2023-08-11T15:37:32.821 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.644+0000 7f1f889fc700 1 --2- 172.21.15.32:0/560812998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8407a2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be23d187ca95f98a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:32.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.644+0000 7f1f837fe700 1 -- 172.21.15.32:0/560812998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78004160 con 0x7f1f8407be80 2023-08-11T15:37:32.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.644+0000 7f1f837fe700 1 -- 172.21.15.32:0/560812998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f780042c0 con 0x7f1f8407be80 2023-08-11T15:37:32.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.644+0000 7f1f837fe700 1 -- 172.21.15.32:0/560812998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78011a00 con 0x7f1f8407be80 2023-08-11T15:37:32.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.645+0000 7f1f899fe700 1 -- 172.21.15.32:0/560812998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 msgr2=0x7f1f8407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:32.822 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.645+0000 7f1f899fe700 1 --2- 172.21.15.32:0/560812998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8407a2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:32.823 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.645+0000 7f1f899fe700 1 -- 172.21.15.32:0/560812998 shutdown_connections 2023-08-11T15:37:32.823 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.645+0000 7f1f899fe700 1 --2- 172.21.15.32:0/560812998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8407a2e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:32.823 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.645+0000 7f1f899fe700 1 -- 172.21.15.32:0/560812998 >> 172.21.15.32:0/560812998 conn(0x7f1f84101440 msgr2=0x7f1f84103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:32.823 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.645+0000 7f1f899fe700 1 -- 172.21.15.32:0/560812998 shutdown_connections 2023-08-11T15:37:32.823 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.645+0000 7f1f899fe700 1 -- 172.21.15.32:0/560812998 wait complete. 2023-08-11T15:37:32.823 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.646+0000 7f1f899fe700 1 Processor -- start 2023-08-11T15:37:32.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.646+0000 7f1f899fe700 1 -- start start 2023-08-11T15:37:32.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.647+0000 7f1f899fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:32.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.647+0000 7f1f899fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419c360 con 0x7f1f8407be80 2023-08-11T15:37:32.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.647+0000 7f1f889fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:32.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.647+0000 7f1f889fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8419be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39962/0 (socket says 172.21.15.32:39962) 2023-08-11T15:37:32.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.647+0000 7f1f889fc700 1 -- 172.21.15.32:0/2720702688 learned_addr learned my addr 172.21.15.32:0/2720702688 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:32.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.647+0000 7f1f889fc700 1 -- 172.21.15.32:0/2720702688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009550 con 0x7f1f8407be80 2023-08-11T15:37:32.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.647+0000 7f1f889fc700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8419be20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009130 tx=0x7f1f78011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:32.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.648+0000 7f1f81ffb700 1 -- 172.21.15.32:0/2720702688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78003b70 con 0x7f1f8407be80 2023-08-11T15:37:32.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.648+0000 7f1f81ffb700 1 -- 172.21.15.32:0/2720702688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f78003cd0 con 0x7f1f8407be80 2023-08-11T15:37:32.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.648+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8419c560 con 0x7f1f8407be80 2023-08-11T15:37:32.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.648+0000 7f1f81ffb700 1 -- 172.21.15.32:0/2720702688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780197d0 con 0x7f1f8407be80 2023-08-11T15:37:32.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.648+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8419c980 con 0x7f1f8407be80 2023-08-11T15:37:32.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.649+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f84195e70 con 0x7f1f8407be80 2023-08-11T15:37:32.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.649+0000 7f1f81ffb700 1 -- 172.21.15.32:0/2720702688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f1f7801abb0 con 0x7f1f8407be80 2023-08-11T15:37:32.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.650+0000 7f1f81ffb700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1f74038160 0x7f1f7403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:32.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.650+0000 7f1f81ffb700 1 -- 172.21.15.32:0/2720702688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1f7804c200 con 0x7f1f8407be80 2023-08-11T15:37:32.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.650+0000 7f1f83fff700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1f74038160 0x7f1f7403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:32.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.651+0000 7f1f83fff700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1f74038160 0x7f1f7403a620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f70006fd0 tx=0x7f1f70006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:32.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.654+0000 7f1f81ffb700 1 -- 172.21.15.32:0/2720702688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1f7801a430 con 0x7f1f8407be80 2023-08-11T15:37:32.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:31.861+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f8402cce0 con 0x7f1f74038160 2023-08-11T15:37:32.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.361+0000 7f1f81ffb700 1 -- 172.21.15.32:0/2720702688 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1f8402cce0 con 0x7f1f74038160 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1f74038160 msgr2=0x7f1f7403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1f74038160 0x7f1f7403a620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f70006fd0 tx=0x7f1f70006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 msgr2=0x7f1f8419be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8419be20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009130 tx=0x7f1f78011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 shutdown_connections 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1f74038160 0x7f1f7403a620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 --2- 172.21.15.32:0/2720702688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f8407be80 0x7f1f8419be20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:32.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 >> 172.21.15.32:0/2720702688 conn(0x7f1f84101440 msgr2=0x7f1f84102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:32.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 shutdown_connections 2023-08-11T15:37:32.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:32.364+0000 7f1f899fe700 1 -- 172.21.15.32:0/2720702688 wait complete. 2023-08-11T15:37:33.694 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:33 smithi032 ceph-mon[97103]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:33.694 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:33 smithi032 ceph-mon[97103]: Generating ssh key... 2023-08-11T15:37:33.694 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:33 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:33.695 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:33 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:34.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCveyt56AAdBGlr5uIR9PbFU1HrjNTLGmYMOuBgMLhiin1hXL0hCm8Omig1nJsKaNAzqlcHP9NEweVcI3ayIFxYk8uD1bXLeAnJKZgMbQp1Aeby4ItPuKkorHk/elOneifVhCO276thYt51nBCQ+9G3/g+WUeZ9/waDr2BRt7lxLoqCJ7Pc6sD/8rD7RSzKd65eXZT415g5y8zNBQUrpKr/y7cnkmvE8HEAwRFEmwlyqYkUk2ioRAKOgW5g8H/GyaVtqAxdsZWCdWLOarVHwifLyxr7iJyP+WsZWTna9+r1gZzNe9ghKnvDVAl3o1F6VUcLuU8wzc9K9b7fHmnoHt/vtM/hy7ISr0cYqMBTpXdF1aI05KrRKPrk44qwvkXJ9LddOdC2hWfyVmefzFj4jyUINSNG07YNtebkcuSbzCsiNFI8hAzDeeOFOofyPDrCbr/rjz0GfyEh2zivc7uir3aTleeUWDNHA1Ab5dF+x2hUMnNi9OQJYglcSxVLNOtNFyM= ceph-cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:37:34.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.335+0000 7fdb137a2700 1 Processor -- start 2023-08-11T15:37:34.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.335+0000 7fdb137a2700 1 -- start start 2023-08-11T15:37:34.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.336+0000 7fdb137a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:34.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.336+0000 7fdb137a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c07a8d0 con 0x7fdb0c07bea0 2023-08-11T15:37:34.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.336+0000 7fdb1153e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:34.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.336+0000 7fdb1153e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39966/0 (socket says 172.21.15.32:39966) 2023-08-11T15:37:34.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.336+0000 7fdb1153e700 1 -- 172.21.15.32:0/3713462646 learned_addr learned my addr 172.21.15.32:0/3713462646 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:34.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.336+0000 7fdb1153e700 1 -- 172.21.15.32:0/3713462646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c07aa10 con 0x7fdb0c07bea0 2023-08-11T15:37:34.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.337+0000 7fdb1153e700 1 --2- 172.21.15.32:0/3713462646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c07a300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098a0 tx=0x7fdb08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a69573d4a24bea server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:34.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.337+0000 7fdb03fff700 1 -- 172.21.15.32:0/3713462646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb08004160 con 0x7fdb0c07bea0 2023-08-11T15:37:34.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.337+0000 7fdb03fff700 1 -- 172.21.15.32:0/3713462646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb080042c0 con 0x7fdb0c07bea0 2023-08-11T15:37:34.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.337+0000 7fdb03fff700 1 -- 172.21.15.32:0/3713462646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb08011a00 con 0x7fdb0c07bea0 2023-08-11T15:37:34.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.337+0000 7fdb137a2700 1 -- 172.21.15.32:0/3713462646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 msgr2=0x7fdb0c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:34.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.337+0000 7fdb137a2700 1 --2- 172.21.15.32:0/3713462646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c07a300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098a0 tx=0x7fdb08009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:34.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.338+0000 7fdb137a2700 1 -- 172.21.15.32:0/3713462646 shutdown_connections 2023-08-11T15:37:34.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.338+0000 7fdb137a2700 1 --2- 172.21.15.32:0/3713462646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c07a300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:34.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.338+0000 7fdb137a2700 1 -- 172.21.15.32:0/3713462646 >> 172.21.15.32:0/3713462646 conn(0x7fdb0c101610 msgr2=0x7fdb0c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:34.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.338+0000 7fdb137a2700 1 -- 172.21.15.32:0/3713462646 shutdown_connections 2023-08-11T15:37:34.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.338+0000 7fdb137a2700 1 -- 172.21.15.32:0/3713462646 wait complete. 2023-08-11T15:37:34.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.339+0000 7fdb137a2700 1 Processor -- start 2023-08-11T15:37:34.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.339+0000 7fdb137a2700 1 -- start start 2023-08-11T15:37:34.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.339+0000 7fdb137a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:34.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.339+0000 7fdb137a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c19c570 con 0x7fdb0c07bea0 2023-08-11T15:37:34.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.339+0000 7fdb1153e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:34.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.339+0000 7fdb1153e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c19c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39970/0 (socket says 172.21.15.32:39970) 2023-08-11T15:37:34.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.340+0000 7fdb1153e700 1 -- 172.21.15.32:0/1671584247 learned_addr learned my addr 172.21.15.32:0/1671584247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:34.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.340+0000 7fdb1153e700 1 -- 172.21.15.32:0/1671584247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08009550 con 0x7fdb0c07bea0 2023-08-11T15:37:34.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.340+0000 7fdb1153e700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c19c030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009130 tx=0x7fdb08011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.340+0000 7fdb027fc700 1 -- 172.21.15.32:0/1671584247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb08003ab0 con 0x7fdb0c07bea0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.340+0000 7fdb027fc700 1 -- 172.21.15.32:0/1671584247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb08003c10 con 0x7fdb0c07bea0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.340+0000 7fdb027fc700 1 -- 172.21.15.32:0/1671584247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb080247d0 con 0x7fdb0c07bea0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.340+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c19c770 con 0x7fdb0c07bea0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.341+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c19cb90 con 0x7fdb0c07bea0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.342+0000 7fdb027fc700 1 -- 172.21.15.32:0/1671584247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fdb0801a430 con 0x7fdb0c07bea0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.342+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c196050 con 0x7fdb0c07bea0 2023-08-11T15:37:34.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.342+0000 7fdb027fc700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fdaf8038160 0x7fdaf803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:34.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.342+0000 7fdb027fc700 1 -- 172.21.15.32:0/1671584247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdb0804c120 con 0x7fdb0c07bea0 2023-08-11T15:37:34.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.342+0000 7fdb10d3d700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fdaf8038160 0x7fdaf803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:34.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.343+0000 7fdb10d3d700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fdaf8038160 0x7fdaf803a620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006fd0 tx=0x7fdafc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:34.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.348+0000 7fdb027fc700 1 -- 172.21.15.32:0/1671584247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fdb08018b40 con 0x7fdb0c07bea0 2023-08-11T15:37:34.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.560+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb0c02cd20 con 0x7fdaf8038160 2023-08-11T15:37:34.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.561+0000 7fdb027fc700 1 -- 172.21.15.32:0/1671584247 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fdb0c02cd20 con 0x7fdaf8038160 2023-08-11T15:37:34.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fdaf8038160 msgr2=0x7fdaf803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:34.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fdaf8038160 0x7fdaf803a620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006fd0 tx=0x7fdafc006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:34.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 msgr2=0x7fdb0c19c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:34.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c19c030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009130 tx=0x7fdb08011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:34.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 shutdown_connections 2023-08-11T15:37:34.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7fdaf8038160 0x7fdaf803a620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:34.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 --2- 172.21.15.32:0/1671584247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c07bea0 0x7fdb0c19c030 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:34.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 >> 172.21.15.32:0/1671584247 conn(0x7fdb0c101610 msgr2=0x7fdb0c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:34.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 shutdown_connections 2023-08-11T15:37:34.017 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:33.564+0000 7fdb137a2700 1 -- 172.21.15.32:0/1671584247 wait complete. 2023-08-11T15:37:34.017 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-08-11T15:37:34.017 INFO:teuthology.orchestra.run.smithi032.stdout:Adding key to root@localhost authorized_keys... 2023-08-11T15:37:34.017 INFO:teuthology.orchestra.run.smithi032.stdout:Adding host smithi032... 2023-08-11T15:37:34.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:34 smithi032 ceph-mon[97103]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:35.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:35 smithi032 ceph-mon[97103]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:37.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:36 smithi032 ceph-mon[97103]: Deploying cephadm binary to smithi032 2023-08-11T15:37:38.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Added host 'smithi032' with addr '172.21.15.32' 2023-08-11T15:37:38.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.546+0000 7f5baae10700 1 Processor -- start 2023-08-11T15:37:38.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.546+0000 7f5baae10700 1 -- start start 2023-08-11T15:37:38.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.546+0000 7f5baae10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:38.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.546+0000 7f5baae10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba407a8f0 con 0x7f5ba407bec0 2023-08-11T15:37:38.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba8bac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:38.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba8bac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48466/0 (socket says 172.21.15.32:48466) 2023-08-11T15:37:38.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba8bac700 1 -- 172.21.15.32:0/738017561 learned_addr learned my addr 172.21.15.32:0/738017561 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:38.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba8bac700 1 -- 172.21.15.32:0/738017561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba407aa30 con 0x7f5ba407bec0 2023-08-11T15:37:38.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba8bac700 1 --2- 172.21.15.32:0/738017561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba407a320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b980098a0 tx=0x7f5b98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57417133b5d9b522 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:38.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba37fe700 1 -- 172.21.15.32:0/738017561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b98004160 con 0x7f5ba407bec0 2023-08-11T15:37:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba37fe700 1 -- 172.21.15.32:0/738017561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b980042c0 con 0x7f5ba407bec0 2023-08-11T15:37:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.547+0000 7f5ba37fe700 1 -- 172.21.15.32:0/738017561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b98011a00 con 0x7f5ba407bec0 2023-08-11T15:37:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.548+0000 7f5baae10700 1 -- 172.21.15.32:0/738017561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 msgr2=0x7f5ba407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.548+0000 7f5baae10700 1 --2- 172.21.15.32:0/738017561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba407a320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b980098a0 tx=0x7f5b98009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.548+0000 7f5baae10700 1 -- 172.21.15.32:0/738017561 shutdown_connections 2023-08-11T15:37:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.548+0000 7f5baae10700 1 --2- 172.21.15.32:0/738017561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba407a320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.548+0000 7f5baae10700 1 -- 172.21.15.32:0/738017561 >> 172.21.15.32:0/738017561 conn(0x7f5ba4101610 msgr2=0x7f5ba4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:38.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.548+0000 7f5baae10700 1 -- 172.21.15.32:0/738017561 shutdown_connections 2023-08-11T15:37:38.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.548+0000 7f5baae10700 1 -- 172.21.15.32:0/738017561 wait complete. 2023-08-11T15:37:38.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.549+0000 7f5baae10700 1 Processor -- start 2023-08-11T15:37:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.550+0000 7f5baae10700 1 -- start start 2023-08-11T15:37:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.550+0000 7f5baae10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.550+0000 7f5baae10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba419c540 con 0x7f5ba407bec0 2023-08-11T15:37:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.550+0000 7f5ba8bac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.550+0000 7f5ba8bac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48482/0 (socket says 172.21.15.32:48482) 2023-08-11T15:37:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.550+0000 7f5ba8bac700 1 -- 172.21.15.32:0/353367771 learned_addr learned my addr 172.21.15.32:0/353367771 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:38.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.550+0000 7f5ba8bac700 1 -- 172.21.15.32:0/353367771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b98009550 con 0x7f5ba407bec0 2023-08-11T15:37:38.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.551+0000 7f5ba8bac700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba419c000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fb0 tx=0x7f5b98011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:38.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.551+0000 7f5ba1ffb700 1 -- 172.21.15.32:0/353367771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b98003a40 con 0x7f5ba407bec0 2023-08-11T15:37:38.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.551+0000 7f5ba1ffb700 1 -- 172.21.15.32:0/353367771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b98003ba0 con 0x7f5ba407bec0 2023-08-11T15:37:38.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.551+0000 7f5ba1ffb700 1 -- 172.21.15.32:0/353367771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b9801a750 con 0x7f5ba407bec0 2023-08-11T15:37:38.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.551+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba419c740 con 0x7f5ba407bec0 2023-08-11T15:37:38.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.551+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba419cb60 con 0x7f5ba407bec0 2023-08-11T15:37:38.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.552+0000 7f5ba1ffb700 1 -- 172.21.15.32:0/353367771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f5b98024b60 con 0x7f5ba407bec0 2023-08-11T15:37:38.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.552+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba4196050 con 0x7f5ba407bec0 2023-08-11T15:37:38.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.553+0000 7f5ba1ffb700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f5b94038160 0x7f5b9403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:38.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.553+0000 7f5ba1ffb700 1 -- 172.21.15.32:0/353367771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5b9804bfc0 con 0x7f5ba407bec0 2023-08-11T15:37:38.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.553+0000 7f5ba3fff700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f5b94038160 0x7f5b9403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:38.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.553+0000 7f5ba3fff700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f5b94038160 0x7f5b9403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5b90006fd0 tx=0x7f5b90006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:38.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.558+0000 7f5ba1ffb700 1 -- 172.21.15.32:0/353367771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5b98024430 con 0x7f5ba407bec0 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:34.770+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ba402cd20 con 0x7f5b94038160 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.043+0000 7f5ba1ffb700 1 -- 172.21.15.32:0/353367771 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f5ba402cd20 con 0x7f5b94038160 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f5b94038160 msgr2=0x7f5b9403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f5b94038160 0x7f5b9403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5b90006fd0 tx=0x7f5b90006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 msgr2=0x7f5ba419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba419c000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fb0 tx=0x7f5b98011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 shutdown_connections 2023-08-11T15:37:38.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f5b94038160 0x7f5b9403a620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:38.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 --2- 172.21.15.32:0/353367771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ba407bec0 0x7f5ba419c000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:38.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.046+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 >> 172.21.15.32:0/353367771 conn(0x7f5ba4101610 msgr2=0x7f5ba4102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:38.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.047+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 shutdown_connections 2023-08-11T15:37:38.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:38.047+0000 7f5baae10700 1 -- 172.21.15.32:0/353367771 wait complete. 2023-08-11T15:37:38.503 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mon service... 2023-08-11T15:37:39.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:39 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:39.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:39 smithi032 ceph-mon[97103]: Added host smithi032 2023-08-11T15:37:39.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:39 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:39.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-08-11T15:37:39.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.040+0000 7f3ec41dd700 1 Processor -- start 2023-08-11T15:37:39.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.041+0000 7f3ec41dd700 1 -- start start 2023-08-11T15:37:39.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.041+0000 7f3ec41dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc105390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:39.697 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.041+0000 7f3ec41dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc105960 con 0x7f3ebc104f70 2023-08-11T15:37:39.697 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.042+0000 7f3ec1f79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:39.697 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.042+0000 7f3ec1f79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc105390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48484/0 (socket says 172.21.15.32:48484) 2023-08-11T15:37:39.697 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.042+0000 7f3ec1f79700 1 -- 172.21.15.32:0/677543006 learned_addr learned my addr 172.21.15.32:0/677543006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:39.697 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.042+0000 7f3ec1f79700 1 -- 172.21.15.32:0/677543006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc106170 con 0x7f3ebc104f70 2023-08-11T15:37:39.697 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.042+0000 7f3ec1f79700 1 --2- 172.21.15.32:0/677543006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc105390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800bb40 tx=0x7f3eb800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59106d3b288eec server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:39.697 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec0f77700 1 -- 172.21.15.32:0/677543006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb800dbe0 con 0x7f3ebc104f70 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec0f77700 1 -- 172.21.15.32:0/677543006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3eb80044d0 con 0x7f3ebc104f70 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec0f77700 1 -- 172.21.15.32:0/677543006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb8010470 con 0x7f3ebc104f70 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec41dd700 1 -- 172.21.15.32:0/677543006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 msgr2=0x7f3ebc105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec41dd700 1 --2- 172.21.15.32:0/677543006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc105390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800bb40 tx=0x7f3eb800be50 comp rx=0 tx=0).stop 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec41dd700 1 -- 172.21.15.32:0/677543006 shutdown_connections 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec41dd700 1 --2- 172.21.15.32:0/677543006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc105390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec41dd700 1 -- 172.21.15.32:0/677543006 >> 172.21.15.32:0/677543006 conn(0x7f3ebc100510 msgr2=0x7f3ebc102950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:39.698 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.046+0000 7f3ec41dd700 1 -- 172.21.15.32:0/677543006 shutdown_connections 2023-08-11T15:37:39.699 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.047+0000 7f3ec41dd700 1 -- 172.21.15.32:0/677543006 wait complete. 2023-08-11T15:37:39.699 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.048+0000 7f3ec41dd700 1 Processor -- start 2023-08-11T15:37:39.699 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.048+0000 7f3ec41dd700 1 -- start start 2023-08-11T15:37:39.699 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.048+0000 7f3ec41dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:39.699 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.048+0000 7f3ec41dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc19c570 con 0x7f3ebc104f70 2023-08-11T15:37:39.699 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.048+0000 7f3ec1f79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:39.699 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.048+0000 7f3ec1f79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc19c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48498/0 (socket says 172.21.15.32:48498) 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.048+0000 7f3ec1f79700 1 -- 172.21.15.32:0/2591764572 learned_addr learned my addr 172.21.15.32:0/2591764572 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.049+0000 7f3ec1f79700 1 -- 172.21.15.32:0/2591764572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb800b7f0 con 0x7f3ebc104f70 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.049+0000 7f3ec1f79700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc19c030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800d650 tx=0x7f3eb8003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.049+0000 7f3eb2ffd700 1 -- 172.21.15.32:0/2591764572 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb80042e0 con 0x7f3ebc104f70 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.049+0000 7f3eb2ffd700 1 -- 172.21.15.32:0/2591764572 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3eb8010bd0 con 0x7f3ebc104f70 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.049+0000 7f3eb2ffd700 1 -- 172.21.15.32:0/2591764572 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb8019c30 con 0x7f3ebc104f70 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.049+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc19c770 con 0x7f3ebc104f70 2023-08-11T15:37:39.700 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.049+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc19cb30 con 0x7f3ebc104f70 2023-08-11T15:37:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.051+0000 7f3eb2ffd700 1 -- 172.21.15.32:0/2591764572 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f3eb8010730 con 0x7f3ebc104f70 2023-08-11T15:37:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.051+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc195c10 con 0x7f3ebc104f70 2023-08-11T15:37:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.051+0000 7f3eb2ffd700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f3ea8038110 0x7f3ea803a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.051+0000 7f3eb2ffd700 1 -- 172.21.15.32:0/2591764572 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3eb804c1e0 con 0x7f3ebc104f70 2023-08-11T15:37:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.051+0000 7f3ec1778700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f3ea8038110 0x7f3ea803a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.052+0000 7f3ec1778700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f3ea8038110 0x7f3ea803a5d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.056+0000 7f3eb2ffd700 1 -- 172.21.15.32:0/2591764572 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3eb800f5b0 con 0x7f3ebc104f70 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.280+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3ebc02cce0 con 0x7f3ea8038110 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.285+0000 7f3eb2ffd700 1 -- 172.21.15.32:0/2591764572 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3ebc02cce0 con 0x7f3ea8038110 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f3ea8038110 msgr2=0x7f3ea803a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f3ea8038110 0x7f3ea803a5d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 msgr2=0x7f3ebc19c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc19c030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800d650 tx=0x7f3eb8003df0 comp rx=0 tx=0).stop 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 shutdown_connections 2023-08-11T15:37:39.702 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f3ea8038110 0x7f3ea803a5d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:39.703 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 --2- 172.21.15.32:0/2591764572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ebc104f70 0x7f3ebc19c030 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:39.703 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 >> 172.21.15.32:0/2591764572 conn(0x7f3ebc100510 msgr2=0x7f3ebc192d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:39.703 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 shutdown_connections 2023-08-11T15:37:39.703 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:39.288+0000 7f3ec41dd700 1 -- 172.21.15.32:0/2591764572 wait complete. 2023-08-11T15:37:39.703 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mgr service... 2023-08-11T15:37:40.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:40 smithi032 ceph-mon[97103]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:40.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:40 smithi032 ceph-mon[97103]: Saving service mon spec with placement count:5 2023-08-11T15:37:40.384 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:40 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:41.028 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-08-11T15:37:41.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.356+0000 7f1c1d648700 1 Processor -- start 2023-08-11T15:37:41.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.356+0000 7f1c1d648700 1 -- start start 2023-08-11T15:37:41.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.356+0000 7f1c1d648700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c18106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:41.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.356+0000 7f1c1d648700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c18106a60 con 0x7f1c18106070 2023-08-11T15:37:41.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.356+0000 7f1c16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c18106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:41.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.356+0000 7f1c16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c18106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48500/0 (socket says 172.21.15.32:48500) 2023-08-11T15:37:41.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.356+0000 7f1c16ffd700 1 -- 172.21.15.32:0/581117245 learned_addr learned my addr 172.21.15.32:0/581117245 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.357+0000 7f1c16ffd700 1 -- 172.21.15.32:0/581117245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c18107270 con 0x7f1c18106070 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.357+0000 7f1c16ffd700 1 --2- 172.21.15.32:0/581117245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c18106490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1c080098a0 tx=0x7f1c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d459e294e0f12861 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.357+0000 7f1c15ffb700 1 -- 172.21.15.32:0/581117245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c08004160 con 0x7f1c18106070 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.357+0000 7f1c15ffb700 1 -- 172.21.15.32:0/581117245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c080042c0 con 0x7f1c18106070 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.357+0000 7f1c15ffb700 1 -- 172.21.15.32:0/581117245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c08011a00 con 0x7f1c18106070 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.358+0000 7f1c1d648700 1 -- 172.21.15.32:0/581117245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 msgr2=0x7f1c18106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.358+0000 7f1c1d648700 1 --2- 172.21.15.32:0/581117245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c18106490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1c080098a0 tx=0x7f1c08009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:41.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.358+0000 7f1c1d648700 1 -- 172.21.15.32:0/581117245 shutdown_connections 2023-08-11T15:37:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.358+0000 7f1c1d648700 1 --2- 172.21.15.32:0/581117245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c18106490 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.358+0000 7f1c1d648700 1 -- 172.21.15.32:0/581117245 >> 172.21.15.32:0/581117245 conn(0x7f1c18101610 msgr2=0x7f1c18103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.358+0000 7f1c1d648700 1 -- 172.21.15.32:0/581117245 shutdown_connections 2023-08-11T15:37:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.358+0000 7f1c1d648700 1 -- 172.21.15.32:0/581117245 wait complete. 2023-08-11T15:37:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.359+0000 7f1c1d648700 1 Processor -- start 2023-08-11T15:37:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.359+0000 7f1c1d648700 1 -- start start 2023-08-11T15:37:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.360+0000 7f1c1d648700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c1819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.360+0000 7f1c1d648700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1819c590 con 0x7f1c18106070 2023-08-11T15:37:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.360+0000 7f1c16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c1819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.360+0000 7f1c16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c1819c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48512/0 (socket says 172.21.15.32:48512) 2023-08-11T15:37:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.360+0000 7f1c16ffd700 1 -- 172.21.15.32:0/1353138938 learned_addr learned my addr 172.21.15.32:0/1353138938 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.360+0000 7f1c16ffd700 1 -- 172.21.15.32:0/1353138938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08009550 con 0x7f1c18106070 2023-08-11T15:37:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.360+0000 7f1c16ffd700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c1819c050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009130 tx=0x7f1c08011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.361+0000 7f1bfffff700 1 -- 172.21.15.32:0/1353138938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c08003ab0 con 0x7f1c18106070 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.361+0000 7f1bfffff700 1 -- 172.21.15.32:0/1353138938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c08003c10 con 0x7f1c18106070 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.361+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1819c790 con 0x7f1c18106070 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.361+0000 7f1bfffff700 1 -- 172.21.15.32:0/1353138938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c080247d0 con 0x7f1c18106070 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.361+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1819cbb0 con 0x7f1c18106070 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.362+0000 7f1bfffff700 1 -- 172.21.15.32:0/1353138938 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f1c0801abb0 con 0x7f1c18106070 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.362+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c18196250 con 0x7f1c18106070 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.363+0000 7f1bfffff700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1c00038110 0x7f1c0003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:41.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.363+0000 7f1bfffff700 1 -- 172.21.15.32:0/1353138938 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1c0804c160 con 0x7f1c18106070 2023-08-11T15:37:41.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.363+0000 7f1c167fc700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1c00038110 0x7f1c0003a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:41.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.363+0000 7f1c167fc700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1c00038110 0x7f1c0003a5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006fd0 tx=0x7f1c0c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:41.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.367+0000 7f1bfffff700 1 -- 172.21.15.32:0/1353138938 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1c08018530 con 0x7f1c18106070 2023-08-11T15:37:41.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.590+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1c1802cd20 con 0x7f1c00038110 2023-08-11T15:37:41.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.595+0000 7f1bfffff700 1 -- 172.21.15.32:0/1353138938 <== mgr.14116 v2:172.21.15.32:6800/4253467068 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1c1802cd20 con 0x7f1c00038110 2023-08-11T15:37:41.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1c00038110 msgr2=0x7f1c0003a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:41.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1c00038110 0x7f1c0003a5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006fd0 tx=0x7f1c0c006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 msgr2=0x7f1c1819c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c1819c050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009130 tx=0x7f1c08011ad0 comp rx=0 tx=0).stop 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 shutdown_connections 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1c00038110 0x7f1c0003a5d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 --2- 172.21.15.32:0/1353138938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c18106070 0x7f1c1819c050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 >> 172.21.15.32:0/1353138938 conn(0x7f1c18101610 msgr2=0x7f1c181039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 shutdown_connections 2023-08-11T15:37:41.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:40.597+0000 7f1c1d648700 1 -- 172.21.15.32:0/1353138938 wait complete. 2023-08-11T15:37:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:41 smithi032 ceph-mon[97103]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:41 smithi032 ceph-mon[97103]: Saving service mgr spec with placement count:2 2023-08-11T15:37:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:41 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:41.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:41 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:41.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:41 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.600+0000 7f1844a66700 1 Processor -- start 2023-08-11T15:37:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.600+0000 7f1844a66700 1 -- start start 2023-08-11T15:37:42.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.601+0000 7f1844a66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:42.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.601+0000 7f1844a66700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184007a8b0 con 0x7f184007be80 2023-08-11T15:37:42.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.601+0000 7f183e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:42.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.601+0000 7f183e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48524/0 (socket says 172.21.15.32:48524) 2023-08-11T15:37:42.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.601+0000 7f183e59c700 1 -- 172.21.15.32:0/3382915516 learned_addr learned my addr 172.21.15.32:0/3382915516 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:42.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.601+0000 7f183e59c700 1 -- 172.21.15.32:0/3382915516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184007a9f0 con 0x7f184007be80 2023-08-11T15:37:42.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.602+0000 7f183e59c700 1 --2- 172.21.15.32:0/3382915516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184007a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f18300098a0 tx=0x7f1830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915bb14a703cbfae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:42.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.602+0000 7f183d59a700 1 -- 172.21.15.32:0/3382915516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1830004160 con 0x7f184007be80 2023-08-11T15:37:42.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.602+0000 7f183d59a700 1 -- 172.21.15.32:0/3382915516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18300042c0 con 0x7f184007be80 2023-08-11T15:37:42.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.602+0000 7f183d59a700 1 -- 172.21.15.32:0/3382915516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18300119b0 con 0x7f184007be80 2023-08-11T15:37:42.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.602+0000 7f1844a66700 1 -- 172.21.15.32:0/3382915516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 msgr2=0x7f184007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:42.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.602+0000 7f1844a66700 1 --2- 172.21.15.32:0/3382915516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184007a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f18300098a0 tx=0x7f1830009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:42.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.603+0000 7f1844a66700 1 -- 172.21.15.32:0/3382915516 shutdown_connections 2023-08-11T15:37:42.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.603+0000 7f1844a66700 1 --2- 172.21.15.32:0/3382915516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184007a2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:42.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.603+0000 7f1844a66700 1 -- 172.21.15.32:0/3382915516 >> 172.21.15.32:0/3382915516 conn(0x7f1840101610 msgr2=0x7f1840103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:42.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.603+0000 7f1844a66700 1 -- 172.21.15.32:0/3382915516 shutdown_connections 2023-08-11T15:37:42.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.603+0000 7f1844a66700 1 -- 172.21.15.32:0/3382915516 wait complete. 2023-08-11T15:37:42.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.604+0000 7f1844a66700 1 Processor -- start 2023-08-11T15:37:42.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.604+0000 7f1844a66700 1 -- start start 2023-08-11T15:37:42.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.604+0000 7f1844a66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.604+0000 7f1844a66700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184019c570 con 0x7f184007be80 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.604+0000 7f183e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.604+0000 7f183e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184019c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48538/0 (socket says 172.21.15.32:48538) 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.604+0000 7f183e59c700 1 -- 172.21.15.32:0/4063307828 learned_addr learned my addr 172.21.15.32:0/4063307828 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.605+0000 7f183e59c700 1 -- 172.21.15.32:0/4063307828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830009550 con 0x7f184007be80 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.605+0000 7f183e59c700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184019c030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1830003f00 tx=0x7f1830011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.605+0000 7f182f7fe700 1 -- 172.21.15.32:0/4063307828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1830003b20 con 0x7f184007be80 2023-08-11T15:37:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.605+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184019c770 con 0x7f184007be80 2023-08-11T15:37:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.605+0000 7f182f7fe700 1 -- 172.21.15.32:0/4063307828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1830003c80 con 0x7f184007be80 2023-08-11T15:37:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.605+0000 7f182f7fe700 1 -- 172.21.15.32:0/4063307828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1830019770 con 0x7f184007be80 2023-08-11T15:37:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.606+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184019cb90 con 0x7f184007be80 2023-08-11T15:37:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.607+0000 7f182f7fe700 1 -- 172.21.15.32:0/4063307828 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f183001a430 con 0x7f184007be80 2023-08-11T15:37:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.607+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18401960b0 con 0x7f184007be80 2023-08-11T15:37:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.607+0000 7f182f7fe700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1828038110 0x7f182803a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.607+0000 7f182f7fe700 1 -- 172.21.15.32:0/4063307828 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f183004c2b0 con 0x7f184007be80 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.607+0000 7f183dd9b700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1828038110 0x7f182803a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.608+0000 7f183dd9b700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1828038110 0x7f182803a5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1834006fd0 tx=0x7f1834006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.612+0000 7f182f7fe700 1 -- 172.21.15.32:0/4063307828 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f183002d960 con 0x7f184007be80 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.826+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f184010c4b0 con 0x7f184007be80 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.830+0000 7f182f7fe700 1 -- 172.21.15.32:0/4063307828 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f183002d960 con 0x7f184007be80 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1828038110 msgr2=0x7f182803a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1828038110 0x7f182803a5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1834006fd0 tx=0x7f1834006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:42.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 msgr2=0x7f184019c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184019c030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1830003f00 tx=0x7f1830011a30 comp rx=0 tx=0).stop 2023-08-11T15:37:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 shutdown_connections 2023-08-11T15:37:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f1828038110 0x7f182803a5d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 --2- 172.21.15.32:0/4063307828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f184007be80 0x7f184019c030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 >> 172.21.15.32:0/4063307828 conn(0x7f1840101610 msgr2=0x7f1840102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 shutdown_connections 2023-08-11T15:37:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:41.833+0000 7f1844a66700 1 -- 172.21.15.32:0/4063307828 wait complete. 2023-08-11T15:37:43.125 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:42 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4063307828' entity='client.admin' 2023-08-11T15:37:43.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.868+0000 7f911ebe7700 1 Processor -- start 2023-08-11T15:37:43.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.868+0000 7f911ebe7700 1 -- start start 2023-08-11T15:37:43.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.868+0000 7f911ebe7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:43.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.868+0000 7f911ebe7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911807a8b0 con 0x7f911807be80 2023-08-11T15:37:43.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.869+0000 7f911c983700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:43.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.869+0000 7f911c983700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48550/0 (socket says 172.21.15.32:48550) 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.869+0000 7f911c983700 1 -- 172.21.15.32:0/1530621062 learned_addr learned my addr 172.21.15.32:0/1530621062 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.869+0000 7f911c983700 1 -- 172.21.15.32:0/1530621062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911807a9f0 con 0x7f911807be80 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.869+0000 7f911c983700 1 --2- 172.21.15.32:0/1530621062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911807a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f910c0098a0 tx=0x7f910c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c139cad46ea335 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f91177fe700 1 -- 172.21.15.32:0/1530621062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f910c004160 con 0x7f911807be80 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f91177fe700 1 -- 172.21.15.32:0/1530621062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f910c0042c0 con 0x7f911807be80 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f91177fe700 1 -- 172.21.15.32:0/1530621062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f910c0119b0 con 0x7f911807be80 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f911ebe7700 1 -- 172.21.15.32:0/1530621062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 msgr2=0x7f911807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:43.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f911ebe7700 1 --2- 172.21.15.32:0/1530621062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911807a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f910c0098a0 tx=0x7f910c009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f911ebe7700 1 -- 172.21.15.32:0/1530621062 shutdown_connections 2023-08-11T15:37:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f911ebe7700 1 --2- 172.21.15.32:0/1530621062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911807a2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.870+0000 7f911ebe7700 1 -- 172.21.15.32:0/1530621062 >> 172.21.15.32:0/1530621062 conn(0x7f9118101610 msgr2=0x7f9118103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.871+0000 7f911ebe7700 1 -- 172.21.15.32:0/1530621062 shutdown_connections 2023-08-11T15:37:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.871+0000 7f911ebe7700 1 -- 172.21.15.32:0/1530621062 wait complete. 2023-08-11T15:37:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.872+0000 7f911ebe7700 1 Processor -- start 2023-08-11T15:37:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.872+0000 7f911ebe7700 1 -- start start 2023-08-11T15:37:43.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.873+0000 7f911ebe7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:43.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.873+0000 7f911ebe7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911819c540 con 0x7f911807be80 2023-08-11T15:37:43.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.873+0000 7f911c983700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:43.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.873+0000 7f911c983700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48566/0 (socket says 172.21.15.32:48566) 2023-08-11T15:37:43.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.873+0000 7f911c983700 1 -- 172.21.15.32:0/4199208022 learned_addr learned my addr 172.21.15.32:0/4199208022 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:43.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.874+0000 7f911c983700 1 -- 172.21.15.32:0/4199208022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c009550 con 0x7f911807be80 2023-08-11T15:37:43.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.874+0000 7f911c983700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911819c000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f910c003f00 tx=0x7f910c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.874+0000 7f9115ffb700 1 -- 172.21.15.32:0/4199208022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f910c003b20 con 0x7f911807be80 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.874+0000 7f9115ffb700 1 -- 172.21.15.32:0/4199208022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f910c003c80 con 0x7f911807be80 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.874+0000 7f9115ffb700 1 -- 172.21.15.32:0/4199208022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f910c019770 con 0x7f911807be80 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.874+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911819c740 con 0x7f911807be80 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.874+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911819cb60 con 0x7f911807be80 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.875+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911810c4b0 con 0x7f911807be80 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.876+0000 7f9115ffb700 1 -- 172.21.15.32:0/4199208022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f910c01ab60 con 0x7f911807be80 2023-08-11T15:37:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.876+0000 7f9115ffb700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f9108038160 0x7f910803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:43.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.876+0000 7f9115ffb700 1 -- 172.21.15.32:0/4199208022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f910c04c250 con 0x7f911807be80 2023-08-11T15:37:43.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.877+0000 7f9117fff700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f9108038160 0x7f910803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:43.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.877+0000 7f9117fff700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f9108038160 0x7f910803a620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9104006fd0 tx=0x7f9104006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:43.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:42.881+0000 7f9115ffb700 1 -- 172.21.15.32:0/4199208022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f910c01a430 con 0x7f911807be80 2023-08-11T15:37:43.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.094+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f911810c6f0 con 0x7f911807be80 2023-08-11T15:37:43.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.097+0000 7f9115ffb700 1 -- 172.21.15.32:0/4199208022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f910c0172b0 con 0x7f911807be80 2023-08-11T15:37:43.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f9108038160 msgr2=0x7f910803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:43.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f9108038160 0x7f910803a620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9104006fd0 tx=0x7f9104006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:43.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 msgr2=0x7f911819c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:43.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911819c000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f910c003f00 tx=0x7f910c011a30 comp rx=0 tx=0).stop 2023-08-11T15:37:43.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 shutdown_connections 2023-08-11T15:37:43.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f9108038160 0x7f910803a620 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:43.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 --2- 172.21.15.32:0/4199208022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911807be80 0x7f911819c000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:43.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.100+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 >> 172.21.15.32:0/4199208022 conn(0x7f9118101610 msgr2=0x7f9118102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:43.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.101+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 shutdown_connections 2023-08-11T15:37:43.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:43.101+0000 7f911ebe7700 1 -- 172.21.15.32:0/4199208022 wait complete. 2023-08-11T15:37:43.496 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling the dashboard module... 2023-08-11T15:37:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:44 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4199208022' entity='client.admin' 2023-08-11T15:37:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:44 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:45.403 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3054831599' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-08-11T15:37:45.404 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-mon[97103]: from='mgr.14116 172.21.15.32:0/650097128' entity='mgr.x' 2023-08-11T15:37:45.404 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: ignoring --setuser ceph since I am not root 2023-08-11T15:37:45.404 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: ignoring --setgroup ceph since I am not root 2023-08-11T15:37:45.404 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:45.401+0000 7f61673a7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:37:45.564 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.050+0000 7f652e5ed700 1 Processor -- start 2023-08-11T15:37:45.571 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.050+0000 7f652e5ed700 1 -- start start 2023-08-11T15:37:45.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.050+0000 7f652e5ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f6528106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:45.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.050+0000 7f652e5ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6528106a60 con 0x7f6528106070 2023-08-11T15:37:45.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.051+0000 7f6527fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f6528106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:45.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.051+0000 7f6527fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f6528106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48574/0 (socket says 172.21.15.32:48574) 2023-08-11T15:37:45.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.051+0000 7f6527fff700 1 -- 172.21.15.32:0/3534485543 learned_addr learned my addr 172.21.15.32:0/3534485543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:45.572 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.051+0000 7f6527fff700 1 -- 172.21.15.32:0/3534485543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528107270 con 0x7f6528106070 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.052+0000 7f6527fff700 1 --2- 172.21.15.32:0/3534485543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f6528106490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65180098a0 tx=0x7f6518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8a0ccce2775b82 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.052+0000 7f6526ffd700 1 -- 172.21.15.32:0/3534485543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6518004160 con 0x7f6528106070 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.052+0000 7f6526ffd700 1 -- 172.21.15.32:0/3534485543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65180042c0 con 0x7f6528106070 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.052+0000 7f6526ffd700 1 -- 172.21.15.32:0/3534485543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65180119b0 con 0x7f6528106070 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.052+0000 7f652e5ed700 1 -- 172.21.15.32:0/3534485543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 msgr2=0x7f6528106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.052+0000 7f652e5ed700 1 --2- 172.21.15.32:0/3534485543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f6528106490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65180098a0 tx=0x7f6518009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.053+0000 7f652e5ed700 1 -- 172.21.15.32:0/3534485543 shutdown_connections 2023-08-11T15:37:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.053+0000 7f652e5ed700 1 --2- 172.21.15.32:0/3534485543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f6528106490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:45.574 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.053+0000 7f652e5ed700 1 -- 172.21.15.32:0/3534485543 >> 172.21.15.32:0/3534485543 conn(0x7f6528101610 msgr2=0x7f6528103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:45.574 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.053+0000 7f652e5ed700 1 -- 172.21.15.32:0/3534485543 shutdown_connections 2023-08-11T15:37:45.574 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.053+0000 7f652e5ed700 1 -- 172.21.15.32:0/3534485543 wait complete. 2023-08-11T15:37:45.574 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.054+0000 7f652e5ed700 1 Processor -- start 2023-08-11T15:37:45.574 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.054+0000 7f652e5ed700 1 -- start start 2023-08-11T15:37:45.574 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.054+0000 7f652e5ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f652807bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:45.575 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.054+0000 7f652e5ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652807c190 con 0x7f6528106070 2023-08-11T15:37:45.575 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.055+0000 7f6527fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f652807bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:45.575 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.055+0000 7f6527fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f652807bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48582/0 (socket says 172.21.15.32:48582) 2023-08-11T15:37:45.575 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.055+0000 7f6527fff700 1 -- 172.21.15.32:0/3054831599 learned_addr learned my addr 172.21.15.32:0/3054831599 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:45.576 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.055+0000 7f6527fff700 1 -- 172.21.15.32:0/3054831599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6518009550 con 0x7f6528106070 2023-08-11T15:37:45.576 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.055+0000 7f6527fff700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f652807bc50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6518003e60 tx=0x7f6518011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:45.576 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.055+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6518003a60 con 0x7f6528106070 2023-08-11T15:37:45.576 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.055+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6518003bc0 con 0x7f6528106070 2023-08-11T15:37:45.577 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.056+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652807a2a0 con 0x7f6528106070 2023-08-11T15:37:45.577 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.056+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65180196c0 con 0x7f6528106070 2023-08-11T15:37:45.577 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.056+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652807a6c0 con 0x7f6528106070 2023-08-11T15:37:45.578 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.057+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f651801a430 con 0x7f6528106070 2023-08-11T15:37:45.578 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.057+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652807b0d0 con 0x7f6528106070 2023-08-11T15:37:45.578 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.057+0000 7f65257fa700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f6510038140 0x7f651003a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:45.578 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.057+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f651804c1d0 con 0x7f6528106070 2023-08-11T15:37:45.579 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.058+0000 7f65277fe700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f6510038140 0x7f651003a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:45.579 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.058+0000 7f65277fe700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f6510038140 0x7f651003a600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f651c006fd0 tx=0x7f651c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:45.579 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.062+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6518010380 con 0x7f6528106070 2023-08-11T15:37:45.579 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:44.338+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f652810a4a0 con 0x7f6528106070 2023-08-11T15:37:45.580 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.101+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f651801a6e0 con 0x7f6528106070 2023-08-11T15:37:45.580 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.103+0000 7f65257fa700 1 -- 172.21.15.32:0/3054831599 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f651804bb20 con 0x7f6528106070 2023-08-11T15:37:45.580 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f6510038140 msgr2=0x7f651003a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:45.581 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f6510038140 0x7f651003a600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f651c006fd0 tx=0x7f651c006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:45.581 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 msgr2=0x7f652807bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:45.581 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f652807bc50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6518003e60 tx=0x7f6518011a30 comp rx=0 tx=0).stop 2023-08-11T15:37:45.581 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 shutdown_connections 2023-08-11T15:37:45.582 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f6510038140 0x7f651003a600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:45.582 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 --2- 172.21.15.32:0/3054831599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528106070 0x7f652807bc50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:45.582 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 >> 172.21.15.32:0/3054831599 conn(0x7f6528101610 msgr2=0x7f65281039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:45.582 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 shutdown_connections 2023-08-11T15:37:45.582 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:45.107+0000 7f652e5ed700 1 -- 172.21.15.32:0/3054831599 wait complete. 2023-08-11T15:37:45.765 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:45.509+0000 7f61673a7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:37:45.766 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:45.637+0000 7f61673a7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:37:45.766 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:45.764+0000 7f61673a7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:37:46.054 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:46.012+0000 7f61673a7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:37:46.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:46 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3054831599' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-08-11T15:37:46.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:46 smithi032 ceph-mon[97103]: mgrmap e9: x(active, since 17s) 2023-08-11T15:37:46.789 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:46.496+0000 7f61673a7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:37:46.789 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:46.667+0000 7f61673a7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:37:46.789 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:46.787+0000 7f61673a7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:37:46.859 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-08-11T15:37:46.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-08-11T15:37:46.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-11T15:37:46.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-11T15:37:46.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-11T15:37:46.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-08-11T15:37:46.860 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.110+0000 7f80ac385700 1 Processor -- start 2023-08-11T15:37:46.861 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.110+0000 7f80ac385700 1 -- start start 2023-08-11T15:37:46.861 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.111+0000 7f80ac385700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:46.861 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.111+0000 7f80ac385700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a407a8b0 con 0x7f80a407be80 2023-08-11T15:37:46.861 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.111+0000 7f80aa121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:46.861 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.111+0000 7f80aa121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40242/0 (socket says 172.21.15.32:40242) 2023-08-11T15:37:46.861 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.111+0000 7f80aa121700 1 -- 172.21.15.32:0/252573503 learned_addr learned my addr 172.21.15.32:0/252573503 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:46.861 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.112+0000 7f80aa121700 1 -- 172.21.15.32:0/252573503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a407a9f0 con 0x7f80a407be80 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.112+0000 7f80aa121700 1 --2- 172.21.15.32:0/252573503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a407a2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80a00098a0 tx=0x7f80a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661b563d0109d59d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.112+0000 7f80a911f700 1 -- 172.21.15.32:0/252573503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80a0004160 con 0x7f80a407be80 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.112+0000 7f80a911f700 1 -- 172.21.15.32:0/252573503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80a00042c0 con 0x7f80a407be80 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.112+0000 7f80a911f700 1 -- 172.21.15.32:0/252573503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80a00119b0 con 0x7f80a407be80 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.113+0000 7f80ac385700 1 -- 172.21.15.32:0/252573503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 msgr2=0x7f80a407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.113+0000 7f80ac385700 1 --2- 172.21.15.32:0/252573503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a407a2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80a00098a0 tx=0x7f80a0009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.113+0000 7f80ac385700 1 -- 172.21.15.32:0/252573503 shutdown_connections 2023-08-11T15:37:46.862 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.113+0000 7f80ac385700 1 --2- 172.21.15.32:0/252573503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a407a2e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:46.863 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.113+0000 7f80ac385700 1 -- 172.21.15.32:0/252573503 >> 172.21.15.32:0/252573503 conn(0x7f80a4101610 msgr2=0x7f80a4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:46.863 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.113+0000 7f80ac385700 1 -- 172.21.15.32:0/252573503 shutdown_connections 2023-08-11T15:37:46.863 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.113+0000 7f80ac385700 1 -- 172.21.15.32:0/252573503 wait complete. 2023-08-11T15:37:46.863 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.114+0000 7f80ac385700 1 Processor -- start 2023-08-11T15:37:46.863 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.115+0000 7f80ac385700 1 -- start start 2023-08-11T15:37:46.863 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.115+0000 7f80ac385700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:46.863 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.115+0000 7f80ac385700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419c540 con 0x7f80a407be80 2023-08-11T15:37:46.864 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.115+0000 7f80aa121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:46.864 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.115+0000 7f80aa121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40248/0 (socket says 172.21.15.32:40248) 2023-08-11T15:37:46.864 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.115+0000 7f80aa121700 1 -- 172.21.15.32:0/2817058951 learned_addr learned my addr 172.21.15.32:0/2817058951 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:46.864 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.116+0000 7f80aa121700 1 -- 172.21.15.32:0/2817058951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0009550 con 0x7f80a407be80 2023-08-11T15:37:46.864 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.116+0000 7f80aa121700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a419c000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80a0006af0 tx=0x7f80a0011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:46.864 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.116+0000 7f809b7fe700 1 -- 172.21.15.32:0/2817058951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80a0003a40 con 0x7f80a407be80 2023-08-11T15:37:46.865 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.116+0000 7f809b7fe700 1 -- 172.21.15.32:0/2817058951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80a0003ba0 con 0x7f80a407be80 2023-08-11T15:37:46.865 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.116+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a419c740 con 0x7f80a407be80 2023-08-11T15:37:46.865 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.116+0000 7f809b7fe700 1 -- 172.21.15.32:0/2817058951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80a00196c0 con 0x7f80a407be80 2023-08-11T15:37:46.865 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.117+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a419cb60 con 0x7f80a407be80 2023-08-11T15:37:46.865 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.118+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4061f50 con 0x7f80a407be80 2023-08-11T15:37:46.865 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.118+0000 7f809b7fe700 1 -- 172.21.15.32:0/2817058951 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f80a001aa80 con 0x7f80a407be80 2023-08-11T15:37:46.865 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.118+0000 7f809b7fe700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 0x7f809003a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.118+0000 7f809b7fe700 1 -- 172.21.15.32:0/2817058951 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f80a004c030 con 0x7f80a407be80 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.118+0000 7f80a9920700 1 -- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 msgr2=0x7f809003a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.118+0000 7f80a9920700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 0x7f809003a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.127+0000 7f809b7fe700 1 -- 172.21.15.32:0/2817058951 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f80a001a430 con 0x7f80a407be80 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.319+0000 7f80a9920700 1 -- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 msgr2=0x7f809003a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.319+0000 7f80a9920700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 0x7f809003a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.395+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f80a410c4b0 con 0x7f80a407be80 2023-08-11T15:37:46.866 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.396+0000 7f809b7fe700 1 -- 172.21.15.32:0/2817058951 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f80a000b3e0 con 0x7f80a407be80 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 msgr2=0x7f809003a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 0x7f809003a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 msgr2=0x7f80a419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a419c000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80a0006af0 tx=0x7f80a0011a10 comp rx=0 tx=0).stop 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 shutdown_connections 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f8090038160 0x7f809003a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 --2- 172.21.15.32:0/2817058951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80a407be80 0x7f80a419c000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 >> 172.21.15.32:0/2817058951 conn(0x7f80a4101610 msgr2=0x7f80a4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:46.868 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 shutdown_connections 2023-08-11T15:37:46.868 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:46.398+0000 7f80ac385700 1 -- 172.21.15.32:0/2817058951 wait complete. 2023-08-11T15:37:46.868 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2023-08-11T15:37:46.868 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 9... 2023-08-11T15:37:47.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:46.941+0000 7f61673a7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:37:47.517 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:47 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2817058951' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-11T15:37:47.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:47.515+0000 7f61673a7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:37:48.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:48.391+0000 7f61673a7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:37:48.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:48.501+0000 7f61673a7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:37:49.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:49.488+0000 7f61673a7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:37:49.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:49.610+0000 7f61673a7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:37:50.216 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:49.841+0000 7f61673a7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:37:50.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:50.215+0000 7f61673a7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:37:50.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:50.496+0000 7f61673a7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:37:50.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:50.697+0000 7f61673a7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:37:51.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:51 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:51.053+0000 7f61673a7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:37:52.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:52.011+0000 7f61673a7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:37:52.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:52.191+0000 7f61673a7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:37:52.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:52.448+0000 7f61673a7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:37:52.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:52.599+0000 7f61673a7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:37:53.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:52.886+0000 7f61673a7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:37:54.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:37:53 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:37:53.984+0000 7f61673a7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:37:54.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: Active manager daemon x restarted 2023-08-11T15:37:54.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: Activating manager daemon x 2023-08-11T15:37:54.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: osdmap e3: 0 total, 0 up, 0 in 2023-08-11T15:37:54.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: mgrmap e10: x(active, starting, since 0.00494356s) 2023-08-11T15:37:54.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:37:54.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T15:37:54.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T15:37:54.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T15:37:54.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T15:37:54.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: Manager daemon x is now available 2023-08-11T15:37:54.363 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:54.363 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:37:55.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T15:37:55.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T15:37:55.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:55 smithi032 ceph-mon[97103]: mgrmap e11: x(active, since 1.00839s) 2023-08-11T15:37:55.378 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-08-11T15:37:55.378 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-08-11T15:37:55.378 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-11T15:37:55.379 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-08-11T15:37:55.379 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.406+0000 7f0fc004b700 1 Processor -- start 2023-08-11T15:37:55.379 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.406+0000 7f0fc004b700 1 -- start start 2023-08-11T15:37:55.379 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.406+0000 7f0fc004b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb807a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:55.380 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.406+0000 7f0fc004b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb807a910 con 0x7f0fb807bee0 2023-08-11T15:37:55.380 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.406+0000 7f0fbdde7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb807a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:55.380 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.407+0000 7f0fbdde7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb807a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40254/0 (socket says 172.21.15.32:40254) 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.407+0000 7f0fbdde7700 1 -- 172.21.15.32:0/2228760413 learned_addr learned my addr 172.21.15.32:0/2228760413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.407+0000 7f0fbdde7700 1 -- 172.21.15.32:0/2228760413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb807aa50 con 0x7f0fb807bee0 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.407+0000 7f0fbdde7700 1 --2- 172.21.15.32:0/2228760413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb807a340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40098a0 tx=0x7f0fb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4b6be452069e5e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fbcde5700 1 -- 172.21.15.32:0/2228760413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb4004160 con 0x7f0fb807bee0 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fbcde5700 1 -- 172.21.15.32:0/2228760413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb40042c0 con 0x7f0fb807bee0 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fbcde5700 1 -- 172.21.15.32:0/2228760413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb40119b0 con 0x7f0fb807bee0 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fc004b700 1 -- 172.21.15.32:0/2228760413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 msgr2=0x7f0fb807a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fc004b700 1 --2- 172.21.15.32:0/2228760413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb807a340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40098a0 tx=0x7f0fb4009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fc004b700 1 -- 172.21.15.32:0/2228760413 shutdown_connections 2023-08-11T15:37:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fc004b700 1 --2- 172.21.15.32:0/2228760413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb807a340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fc004b700 1 -- 172.21.15.32:0/2228760413 >> 172.21.15.32:0/2228760413 conn(0x7f0fb8101610 msgr2=0x7f0fb8103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fc004b700 1 -- 172.21.15.32:0/2228760413 shutdown_connections 2023-08-11T15:37:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.408+0000 7f0fc004b700 1 -- 172.21.15.32:0/2228760413 wait complete. 2023-08-11T15:37:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.409+0000 7f0fc004b700 1 Processor -- start 2023-08-11T15:37:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.410+0000 7f0fc004b700 1 -- start start 2023-08-11T15:37:55.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.410+0000 7f0fc004b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:55.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.410+0000 7f0fc004b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb819c590 con 0x7f0fb807bee0 2023-08-11T15:37:55.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.410+0000 7f0fbdde7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:55.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.410+0000 7f0fbdde7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb819c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40258/0 (socket says 172.21.15.32:40258) 2023-08-11T15:37:55.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.410+0000 7f0fbdde7700 1 -- 172.21.15.32:0/1393162362 learned_addr learned my addr 172.21.15.32:0/1393162362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:55.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.411+0000 7f0fbdde7700 1 -- 172.21.15.32:0/1393162362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4009550 con 0x7f0fb807bee0 2023-08-11T15:37:55.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.411+0000 7f0fbdde7700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb819c050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006af0 tx=0x7f0fb4011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:55.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.411+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb4003a40 con 0x7f0fb807bee0 2023-08-11T15:37:55.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.411+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb819c790 con 0x7f0fb807bee0 2023-08-11T15:37:55.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.411+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb4003ba0 con 0x7f0fb807bee0 2023-08-11T15:37:55.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.411+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb4019690 con 0x7f0fb807bee0 2023-08-11T15:37:55.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.411+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb819cbb0 con 0x7f0fb807bee0 2023-08-11T15:37:55.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.413+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f0fb401a430 con 0x7f0fb807bee0 2023-08-11T15:37:55.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.413+0000 7f0faeffd700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:55.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.413+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 --> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fa403ad30 con 0x7f0fa4038160 2023-08-11T15:37:55.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.413+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0fb404bf30 con 0x7f0fb807bee0 2023-08-11T15:37:55.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.413+0000 7f0fbd5e6700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 msgr2=0x7f0fa403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:55.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.413+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:37:55.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.614+0000 7f0fbd5e6700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 msgr2=0x7f0fa403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:55.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:47.614+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T15:37:55.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:48.015+0000 7f0fbd5e6700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 msgr2=0x7f0fa403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:55.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:48.015+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-11T15:37:55.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:48.816+0000 7f0fbd5e6700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 msgr2=0x7f0fa403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:55.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:48.816+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-11T15:37:55.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:50.417+0000 7f0fbd5e6700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 msgr2=0x7f0fa403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:55.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:50.418+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-11T15:37:55.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:53.620+0000 7f0fbd5e6700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 msgr2=0x7f0fa403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/4253467068 2023-08-11T15:37:55.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:53.620+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-11T15:37:55.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:53.989+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 10) v1 ==== 44915+0+0 (secure 0 0 0) 0x7f0fb401aa60 con 0x7f0fb807bee0 2023-08-11T15:37:55.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:53.989+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 msgr2=0x7f0fa403a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:37:55.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:53.989+0000 7f0faeffd700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:55.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:54.992+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 11) v1 ==== 45042+0+0 (secure 0 0 0) 0x7f0fb404d850 con 0x7f0fb807bee0 2023-08-11T15:37:55.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:54.992+0000 7f0faeffd700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0fa403bc10 0x7f0fa403e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:55.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:54.992+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fa403ad30 con 0x7f0fa403bc10 2023-08-11T15:37:55.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:54.992+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0fa403bc10 0x7f0fa403e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:55.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:54.993+0000 7f0fbd5e6700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0fa403bc10 0x7f0fa403e000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8003a10 tx=0x7f0fa8009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:55.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:54.995+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f0fa403ad30 con 0x7f0fa403bc10 2023-08-11T15:37:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.003+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0fb819d390 con 0x7f0fa403bc10 2023-08-11T15:37:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.004+0000 7f0faeffd700 1 -- 172.21.15.32:0/1393162362 <== mgr.14148 v2:172.21.15.32:6800/2482310260 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f0fb819d390 con 0x7f0fa403bc10 2023-08-11T15:37:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0fa403bc10 msgr2=0x7f0fa403e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0fa403bc10 0x7f0fa403e000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8003a10 tx=0x7f0fa8009270 comp rx=0 tx=0).stop 2023-08-11T15:37:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 msgr2=0x7f0fb819c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb819c050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006af0 tx=0x7f0fb4011a10 comp rx=0 tx=0).stop 2023-08-11T15:37:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 shutdown_connections 2023-08-11T15:37:55.390 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0fa403bc10 0x7f0fa403e000 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:55.390 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb807bee0 0x7f0fb819c050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:55.390 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 --2- 172.21.15.32:0/1393162362 >> [v2:172.21.15.32:6800/4253467068,v1:172.21.15.32:6801/4253467068] conn(0x7f0fa4038160 0x7f0fa403a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:55.390 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 >> 172.21.15.32:0/1393162362 conn(0x7f0fb8101610 msgr2=0x7f0fb81039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:55.390 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 shutdown_connections 2023-08-11T15:37:55.390 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.005+0000 7f0fc004b700 1 -- 172.21.15.32:0/1393162362 wait complete. 2023-08-11T15:37:55.390 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 9 is available 2023-08-11T15:37:55.391 INFO:teuthology.orchestra.run.smithi032.stdout:Generating a dashboard self-signed certificate... 2023-08-11T15:37:56.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:56 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:54] ENGINE Bus STARTING 2023-08-11T15:37:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:56 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:54] ENGINE Serving on http://172.21.15.32:8765 2023-08-11T15:37:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:56 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:54] ENGINE Serving on https://172.21.15.32:7150 2023-08-11T15:37:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:56 smithi032 ceph-mon[97103]: [11/Aug/2023:15:37:54] ENGINE Bus STARTED 2023-08-11T15:37:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:56 smithi032 ceph-mon[97103]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-08-11T15:37:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:56 smithi032 ceph-mon[97103]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-08-11T15:37:56.662 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-08-11T15:37:56.663 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.940+0000 7ffa396c4700 1 Processor -- start 2023-08-11T15:37:56.663 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.940+0000 7ffa396c4700 1 -- start start 2023-08-11T15:37:56.663 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.940+0000 7ffa396c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:56.663 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.940+0000 7ffa396c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3407a8b0 con 0x7ffa3407be80 2023-08-11T15:37:56.663 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.940+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.941+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48870/0 (socket says 172.21.15.32:48870) 2023-08-11T15:37:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.941+0000 7ffa32ffd700 1 -- 172.21.15.32:0/1881651774 learned_addr learned my addr 172.21.15.32:0/1881651774 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.941+0000 7ffa32ffd700 1 -- 172.21.15.32:0/1881651774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3407a9f0 con 0x7ffa3407be80 2023-08-11T15:37:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.941+0000 7ffa32ffd700 1 --2- 172.21.15.32:0/1881651774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3407a2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa240098a0 tx=0x7ffa24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98368147e3eb9e7d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.941+0000 7ffa31ffb700 1 -- 172.21.15.32:0/1881651774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa24004160 con 0x7ffa3407be80 2023-08-11T15:37:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.941+0000 7ffa31ffb700 1 -- 172.21.15.32:0/1881651774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa240042c0 con 0x7ffa3407be80 2023-08-11T15:37:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.941+0000 7ffa31ffb700 1 -- 172.21.15.32:0/1881651774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa240119b0 con 0x7ffa3407be80 2023-08-11T15:37:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.942+0000 7ffa396c4700 1 -- 172.21.15.32:0/1881651774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 msgr2=0x7ffa3407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.942+0000 7ffa396c4700 1 --2- 172.21.15.32:0/1881651774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3407a2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa240098a0 tx=0x7ffa24009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.942+0000 7ffa396c4700 1 -- 172.21.15.32:0/1881651774 shutdown_connections 2023-08-11T15:37:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.942+0000 7ffa396c4700 1 --2- 172.21.15.32:0/1881651774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3407a2e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.942+0000 7ffa396c4700 1 -- 172.21.15.32:0/1881651774 >> 172.21.15.32:0/1881651774 conn(0x7ffa34101610 msgr2=0x7ffa34103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:56.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.942+0000 7ffa396c4700 1 -- 172.21.15.32:0/1881651774 shutdown_connections 2023-08-11T15:37:56.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.942+0000 7ffa396c4700 1 -- 172.21.15.32:0/1881651774 wait complete. 2023-08-11T15:37:56.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.944+0000 7ffa396c4700 1 Processor -- start 2023-08-11T15:37:56.666 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.944+0000 7ffa396c4700 1 -- start start 2023-08-11T15:37:56.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.944+0000 7ffa396c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:56.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.944+0000 7ffa396c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3419c540 con 0x7ffa3407be80 2023-08-11T15:37:56.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.944+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:56.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.944+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48872/0 (socket says 172.21.15.32:48872) 2023-08-11T15:37:56.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.944+0000 7ffa32ffd700 1 -- 172.21.15.32:0/2737760956 learned_addr learned my addr 172.21.15.32:0/2737760956 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:56.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.945+0000 7ffa32ffd700 1 -- 172.21.15.32:0/2737760956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24009550 con 0x7ffa3407be80 2023-08-11T15:37:56.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.945+0000 7ffa32ffd700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3419c000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa24003f00 tx=0x7ffa24011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:56.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.945+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa24003b20 con 0x7ffa3407be80 2023-08-11T15:37:56.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.945+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3419c740 con 0x7ffa3407be80 2023-08-11T15:37:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.945+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa24003c80 con 0x7ffa3407be80 2023-08-11T15:37:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.945+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa24019770 con 0x7ffa3407be80 2023-08-11T15:37:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.946+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3419cb60 con 0x7ffa3407be80 2023-08-11T15:37:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.947+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 11) v1 ==== 45042+0+0 (secure 0 0 0) 0x7ffa240199e0 con 0x7ffa3407be80 2023-08-11T15:37:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.947+0000 7ffa1bfff700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffa1c038080 0x7ffa1c03a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.947+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa34061f50 con 0x7ffa3407be80 2023-08-11T15:37:56.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.947+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffa2404c210 con 0x7ffa3407be80 2023-08-11T15:37:56.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.947+0000 7ffa327fc700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffa1c038080 0x7ffa1c03a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:56.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.948+0000 7ffa327fc700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffa1c038080 0x7ffa1c03a540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006fd0 tx=0x7ffa28006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:56.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:55.957+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ffa2402d9a0 con 0x7ffa3407be80 2023-08-11T15:37:56.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.044+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7ffa24019d30 con 0x7ffa3407be80 2023-08-11T15:37:56.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.201+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa34103970 con 0x7ffa1c038080 2023-08-11T15:37:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.235+0000 7ffa1bfff700 1 -- 172.21.15.32:0/2737760956 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ffa34103970 con 0x7ffa1c038080 2023-08-11T15:37:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.237+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffa1c038080 msgr2=0x7ffa1c03a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.237+0000 7ffa396c4700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffa1c038080 0x7ffa1c03a540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006fd0 tx=0x7ffa28006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.237+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 msgr2=0x7ffa3419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.237+0000 7ffa396c4700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3419c000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa24003f00 tx=0x7ffa24011a30 comp rx=0 tx=0).stop 2023-08-11T15:37:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.238+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 shutdown_connections 2023-08-11T15:37:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.238+0000 7ffa396c4700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffa1c038080 0x7ffa1c03a540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:56.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.238+0000 7ffa396c4700 1 --2- 172.21.15.32:0/2737760956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3407be80 0x7ffa3419c000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:56.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.238+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 >> 172.21.15.32:0/2737760956 conn(0x7ffa34101610 msgr2=0x7ffa34102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:56.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.238+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 shutdown_connections 2023-08-11T15:37:56.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:56.238+0000 7ffa396c4700 1 -- 172.21.15.32:0/2737760956 wait complete. 2023-08-11T15:37:56.672 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial admin user... 2023-08-11T15:37:57.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:57 smithi032 ceph-mon[97103]: mgrmap e12: x(active, since 2s) 2023-08-11T15:37:57.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:57 smithi032 ceph-mon[97103]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:37:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:37:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:37:58.156 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$VpcYQ32n4w4.Ty7iS72Lq.zXRLt/FqwAehYHCsiYOY9IvWk3yFUU6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1691768277, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-08-11T15:37:58.156 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.217+0000 7fef61647700 1 Processor -- start 2023-08-11T15:37:58.156 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.217+0000 7fef61647700 1 -- start start 2023-08-11T15:37:58.157 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.217+0000 7fef61647700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:58.157 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.217+0000 7fef61647700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c106a60 con 0x7fef5c106070 2023-08-11T15:37:58.157 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.218+0000 7fef5affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:58.157 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.218+0000 7fef5affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48876/0 (socket says 172.21.15.32:48876) 2023-08-11T15:37:58.157 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.218+0000 7fef5affd700 1 -- 172.21.15.32:0/1153714807 learned_addr learned my addr 172.21.15.32:0/1153714807 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:58.157 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.218+0000 7fef5affd700 1 -- 172.21.15.32:0/1153714807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c107270 con 0x7fef5c106070 2023-08-11T15:37:58.158 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.218+0000 7fef5affd700 1 --2- 172.21.15.32:0/1153714807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c106490 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0098a0 tx=0x7fef4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ab64e82e2e7b90 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:58.158 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef59ffb700 1 -- 172.21.15.32:0/1153714807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef4c004160 con 0x7fef5c106070 2023-08-11T15:37:58.158 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef59ffb700 1 -- 172.21.15.32:0/1153714807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4c0042c0 con 0x7fef5c106070 2023-08-11T15:37:58.158 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef59ffb700 1 -- 172.21.15.32:0/1153714807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef4c0119b0 con 0x7fef5c106070 2023-08-11T15:37:58.158 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef61647700 1 -- 172.21.15.32:0/1153714807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 msgr2=0x7fef5c106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:58.158 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef61647700 1 --2- 172.21.15.32:0/1153714807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c106490 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0098a0 tx=0x7fef4c009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:58.159 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef61647700 1 -- 172.21.15.32:0/1153714807 shutdown_connections 2023-08-11T15:37:58.159 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef61647700 1 --2- 172.21.15.32:0/1153714807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c106490 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:58.159 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef61647700 1 -- 172.21.15.32:0/1153714807 >> 172.21.15.32:0/1153714807 conn(0x7fef5c101610 msgr2=0x7fef5c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:58.159 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef61647700 1 -- 172.21.15.32:0/1153714807 shutdown_connections 2023-08-11T15:37:58.159 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.219+0000 7fef61647700 1 -- 172.21.15.32:0/1153714807 wait complete. 2023-08-11T15:37:58.159 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.220+0000 7fef61647700 1 Processor -- start 2023-08-11T15:37:58.160 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.220+0000 7fef61647700 1 -- start start 2023-08-11T15:37:58.160 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.221+0000 7fef61647700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c07bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:58.160 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.221+0000 7fef61647700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c07c190 con 0x7fef5c106070 2023-08-11T15:37:58.160 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.221+0000 7fef5affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c07bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:58.160 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.221+0000 7fef5affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c07bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48882/0 (socket says 172.21.15.32:48882) 2023-08-11T15:37:58.160 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.221+0000 7fef5affd700 1 -- 172.21.15.32:0/3669642091 learned_addr learned my addr 172.21.15.32:0/3669642091 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:58.161 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.221+0000 7fef5affd700 1 -- 172.21.15.32:0/3669642091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c009550 con 0x7fef5c106070 2023-08-11T15:37:58.165 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.222+0000 7fef5affd700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c07bc50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef4c006af0 tx=0x7fef4c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:58.165 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.222+0000 7fef43fff700 1 -- 172.21.15.32:0/3669642091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef4c003a40 con 0x7fef5c106070 2023-08-11T15:37:58.165 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.222+0000 7fef43fff700 1 -- 172.21.15.32:0/3669642091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4c003ba0 con 0x7fef5c106070 2023-08-11T15:37:58.165 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.222+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c07a2a0 con 0x7fef5c106070 2023-08-11T15:37:58.166 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.222+0000 7fef43fff700 1 -- 172.21.15.32:0/3669642091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef4c019690 con 0x7fef5c106070 2023-08-11T15:37:58.166 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.222+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c07a6c0 con 0x7fef5c106070 2023-08-11T15:37:58.166 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.223+0000 7fef43fff700 1 -- 172.21.15.32:0/3669642091 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7fef4c01a430 con 0x7fef5c106070 2023-08-11T15:37:58.166 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.223+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5c07b0d0 con 0x7fef5c106070 2023-08-11T15:37:58.167 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.223+0000 7fef43fff700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fef44038110 0x7fef4403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:58.167 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.223+0000 7fef43fff700 1 -- 172.21.15.32:0/3669642091 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef4c04c500 con 0x7fef5c106070 2023-08-11T15:37:58.167 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.224+0000 7fef5a7fc700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fef44038110 0x7fef4403a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:58.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.224+0000 7fef5a7fc700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fef44038110 0x7fef4403a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef50006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:58.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.229+0000 7fef43fff700 1 -- 172.21.15.32:0/3669642091 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fef4c023570 con 0x7fef5c106070 2023-08-11T15:37:58.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.455+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fef5c061d70 con 0x7fef44038110 2023-08-11T15:37:58.168 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.705+0000 7fef43fff700 1 -- 172.21.15.32:0/3669642091 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fef5c061d70 con 0x7fef44038110 2023-08-11T15:37:58.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fef44038110 msgr2=0x7fef4403a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:58.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fef44038110 0x7fef4403a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef50006fd0 tx=0x7fef50006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:58.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 msgr2=0x7fef5c07bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:58.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c07bc50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef4c006af0 tx=0x7fef4c011a10 comp rx=0 tx=0).stop 2023-08-11T15:37:58.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 shutdown_connections 2023-08-11T15:37:58.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fef44038110 0x7fef4403a5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:58.170 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 --2- 172.21.15.32:0/3669642091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef5c106070 0x7fef5c07bc50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:58.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 >> 172.21.15.32:0/3669642091 conn(0x7fef5c101610 msgr2=0x7fef5c102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:58.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 shutdown_connections 2023-08-11T15:37:58.171 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:57.707+0000 7fef61647700 1 -- 172.21.15.32:0/3669642091 wait complete. 2023-08-11T15:37:58.172 INFO:teuthology.orchestra.run.smithi032.stdout:Fetching dashboard port number... 2023-08-11T15:37:58.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:58 smithi032 ceph-mon[97103]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:37:58.979 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:37:58.979 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:37:59.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 8443 2023-08-11T15:37:59.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.717+0000 7f249e227700 1 Processor -- start 2023-08-11T15:37:59.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.717+0000 7f249e227700 1 -- start start 2023-08-11T15:37:59.434 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.717+0000 7f249e227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f24981056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:59.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.717+0000 7f249e227700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498105c70 con 0x7f2498105280 2023-08-11T15:37:59.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.718+0000 7f24977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f24981056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:59.435 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.718+0000 7f24977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f24981056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48884/0 (socket says 172.21.15.32:48884) 2023-08-11T15:37:59.436 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.718+0000 7f24977fe700 1 -- 172.21.15.32:0/1851873810 learned_addr learned my addr 172.21.15.32:0/1851873810 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:59.436 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.718+0000 7f24977fe700 1 -- 172.21.15.32:0/1851873810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498105db0 con 0x7f2498105280 2023-08-11T15:37:59.436 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.719+0000 7f24977fe700 1 --2- 172.21.15.32:0/1851873810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f24981056a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24880098a0 tx=0x7f2488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356b791ff00d1d58 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:59.437 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.719+0000 7f24967fc700 1 -- 172.21.15.32:0/1851873810 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2488004160 con 0x7f2498105280 2023-08-11T15:37:59.437 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.719+0000 7f24967fc700 1 -- 172.21.15.32:0/1851873810 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24880042c0 con 0x7f2498105280 2023-08-11T15:37:59.437 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.719+0000 7f24967fc700 1 -- 172.21.15.32:0/1851873810 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24880119b0 con 0x7f2498105280 2023-08-11T15:37:59.438 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.720+0000 7f249e227700 1 -- 172.21.15.32:0/1851873810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 msgr2=0x7f24981056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:59.438 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.720+0000 7f249e227700 1 --2- 172.21.15.32:0/1851873810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f24981056a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24880098a0 tx=0x7f2488009bb0 comp rx=0 tx=0).stop 2023-08-11T15:37:59.438 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.720+0000 7f249e227700 1 -- 172.21.15.32:0/1851873810 shutdown_connections 2023-08-11T15:37:59.439 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.720+0000 7f249e227700 1 --2- 172.21.15.32:0/1851873810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f24981056a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:59.439 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.720+0000 7f249e227700 1 -- 172.21.15.32:0/1851873810 >> 172.21.15.32:0/1851873810 conn(0x7f24980ff880 msgr2=0x7f2498101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:59.440 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.720+0000 7f249e227700 1 -- 172.21.15.32:0/1851873810 shutdown_connections 2023-08-11T15:37:59.440 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.720+0000 7f249e227700 1 -- 172.21.15.32:0/1851873810 wait complete. 2023-08-11T15:37:59.440 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.721+0000 7f249e227700 1 Processor -- start 2023-08-11T15:37:59.441 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.721+0000 7f249e227700 1 -- start start 2023-08-11T15:37:59.441 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.721+0000 7f249e227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f249819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:59.441 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.721+0000 7f249e227700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249819c560 con 0x7f2498105280 2023-08-11T15:37:59.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.722+0000 7f24977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f249819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:59.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.722+0000 7f24977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f249819c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48892/0 (socket says 172.21.15.32:48892) 2023-08-11T15:37:59.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.722+0000 7f24977fe700 1 -- 172.21.15.32:0/2868956160 learned_addr learned my addr 172.21.15.32:0/2868956160 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:37:59.442 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.722+0000 7f24977fe700 1 -- 172.21.15.32:0/2868956160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2488009550 con 0x7f2498105280 2023-08-11T15:37:59.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.722+0000 7f24977fe700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f249819c020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2488003f00 tx=0x7f2488011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:59.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.723+0000 7f2494ff9700 1 -- 172.21.15.32:0/2868956160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2488003b20 con 0x7f2498105280 2023-08-11T15:37:59.443 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.723+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249819c760 con 0x7f2498105280 2023-08-11T15:37:59.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.723+0000 7f2494ff9700 1 -- 172.21.15.32:0/2868956160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2488003c80 con 0x7f2498105280 2023-08-11T15:37:59.444 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.723+0000 7f2494ff9700 1 -- 172.21.15.32:0/2868956160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2488019770 con 0x7f2498105280 2023-08-11T15:37:59.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.723+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249819cb80 con 0x7f2498105280 2023-08-11T15:37:59.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.724+0000 7f2494ff9700 1 -- 172.21.15.32:0/2868956160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f24880199e0 con 0x7f2498105280 2023-08-11T15:37:59.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.724+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484005320 con 0x7f2498105280 2023-08-11T15:37:59.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.725+0000 7f2494ff9700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f24800381b0 0x7f248003a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:37:59.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.725+0000 7f2494ff9700 1 -- 172.21.15.32:0/2868956160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f248804c5c0 con 0x7f2498105280 2023-08-11T15:37:59.445 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.725+0000 7f2496ffd700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f24800381b0 0x7f248003a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:37:59.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.726+0000 7f2496ffd700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f24800381b0 0x7f248003a670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f248c006fd0 tx=0x7f248c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:37:59.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.730+0000 7f2494ff9700 1 -- 172.21.15.32:0/2868956160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2488018b80 con 0x7f2498105280 2023-08-11T15:37:59.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.952+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f2484005b50 con 0x7f2498105280 2023-08-11T15:37:59.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.953+0000 7f2494ff9700 1 -- 172.21.15.32:0/2868956160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f2488023e60 con 0x7f2498105280 2023-08-11T15:37:59.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.955+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f24800381b0 msgr2=0x7f248003a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:59.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.955+0000 7f249e227700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f24800381b0 0x7f248003a670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f248c006fd0 tx=0x7f248c006e40 comp rx=0 tx=0).stop 2023-08-11T15:37:59.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.955+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 msgr2=0x7f249819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:37:59.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.955+0000 7f249e227700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f249819c020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2488003f00 tx=0x7f2488011a30 comp rx=0 tx=0).stop 2023-08-11T15:37:59.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.956+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 shutdown_connections 2023-08-11T15:37:59.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.956+0000 7f249e227700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f24800381b0 0x7f248003a670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:59.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.956+0000 7f249e227700 1 --2- 172.21.15.32:0/2868956160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498105280 0x7f249819c020 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:37:59.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.956+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 >> 172.21.15.32:0/2868956160 conn(0x7f24980ff880 msgr2=0x7f2498192960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:37:59.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.956+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 shutdown_connections 2023-08-11T15:37:59.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:37:58.956+0000 7f249e227700 1 -- 172.21.15.32:0/2868956160 wait complete. 2023-08-11T15:37:59.453 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T15:37:59.453 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-08-11T15:37:59.469 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T15:37:59.470 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-08-11T15:37:59.470 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-08-11T15:37:59.470 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-08-11T15:37:59.473 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph Dashboard is now available at: 2023-08-11T15:37:59.473 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:37:59.473 INFO:teuthology.orchestra.run.smithi032.stdout: URL: https://smithi032.front.sepia.ceph.com:8443/ 2023-08-11T15:37:59.474 INFO:teuthology.orchestra.run.smithi032.stdout: User: admin 2023-08-11T15:37:59.474 INFO:teuthology.orchestra.run.smithi032.stdout: Password: e01ojaqlh0 2023-08-11T15:37:59.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:37:59.474 INFO:teuthology.orchestra.run.smithi032.stdout:Saving cluster configuration to /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config directory 2023-08-11T15:37:59.475 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling autotune for osd_memory_target 2023-08-11T15:38:00.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:37:59 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2868956160' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-08-11T15:38:00.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.163+0000 7f0e4a446700 1 Processor -- start 2023-08-11T15:38:00.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.163+0000 7f0e4a446700 1 -- start start 2023-08-11T15:38:00.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.164+0000 7f0e4a446700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:00.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.164+0000 7f0e4a446700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4407a8d0 con 0x7f0e4407bea0 2023-08-11T15:38:00.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.164+0000 7f0e43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:00.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.164+0000 7f0e43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48896/0 (socket says 172.21.15.32:48896) 2023-08-11T15:38:00.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.164+0000 7f0e43fff700 1 -- 172.21.15.32:0/1306407662 learned_addr learned my addr 172.21.15.32:0/1306407662 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:00.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.164+0000 7f0e43fff700 1 -- 172.21.15.32:0/1306407662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4407aa10 con 0x7f0e4407bea0 2023-08-11T15:38:00.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.165+0000 7f0e43fff700 1 --2- 172.21.15.32:0/1306407662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4407a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098a0 tx=0x7f0e34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a774195fd08ded0f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:00.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.165+0000 7f0e42ffd700 1 -- 172.21.15.32:0/1306407662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e34004160 con 0x7f0e4407bea0 2023-08-11T15:38:00.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.165+0000 7f0e42ffd700 1 -- 172.21.15.32:0/1306407662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e340042c0 con 0x7f0e4407bea0 2023-08-11T15:38:00.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.165+0000 7f0e42ffd700 1 -- 172.21.15.32:0/1306407662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e340119b0 con 0x7f0e4407bea0 2023-08-11T15:38:00.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.166+0000 7f0e4a446700 1 -- 172.21.15.32:0/1306407662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 msgr2=0x7f0e4407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:00.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.166+0000 7f0e4a446700 1 --2- 172.21.15.32:0/1306407662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4407a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098a0 tx=0x7f0e34009bb0 comp rx=0 tx=0).stop 2023-08-11T15:38:00.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.166+0000 7f0e4a446700 1 -- 172.21.15.32:0/1306407662 shutdown_connections 2023-08-11T15:38:00.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.166+0000 7f0e4a446700 1 --2- 172.21.15.32:0/1306407662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4407a300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:00.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.166+0000 7f0e4a446700 1 -- 172.21.15.32:0/1306407662 >> 172.21.15.32:0/1306407662 conn(0x7f0e44101610 msgr2=0x7f0e44103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:00.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.166+0000 7f0e4a446700 1 -- 172.21.15.32:0/1306407662 shutdown_connections 2023-08-11T15:38:00.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.166+0000 7f0e4a446700 1 -- 172.21.15.32:0/1306407662 wait complete. 2023-08-11T15:38:00.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.167+0000 7f0e4a446700 1 Processor -- start 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.167+0000 7f0e4a446700 1 -- start start 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.167+0000 7f0e4a446700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.167+0000 7f0e4a446700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4419c590 con 0x7f0e4407bea0 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4419c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48910/0 (socket says 172.21.15.32:48910) 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e43fff700 1 -- 172.21.15.32:0/63329804 learned_addr learned my addr 172.21.15.32:0/63329804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e43fff700 1 -- 172.21.15.32:0/63329804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009550 con 0x7f0e4407bea0 2023-08-11T15:38:00.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e43fff700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4419c050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0e34003e60 tx=0x7f0e34011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e417fa700 1 -- 172.21.15.32:0/63329804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e34003a60 con 0x7f0e4407bea0 2023-08-11T15:38:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e417fa700 1 -- 172.21.15.32:0/63329804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e34003bc0 con 0x7f0e4407bea0 2023-08-11T15:38:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4419c790 con 0x7f0e4407bea0 2023-08-11T15:38:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.168+0000 7f0e417fa700 1 -- 172.21.15.32:0/63329804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e340196c0 con 0x7f0e4407bea0 2023-08-11T15:38:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.169+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4419cbb0 con 0x7f0e4407bea0 2023-08-11T15:38:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.169+0000 7f0e417fa700 1 -- 172.21.15.32:0/63329804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f0e3401a430 con 0x7f0e4407bea0 2023-08-11T15:38:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.170+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44196070 con 0x7f0e4407bea0 2023-08-11T15:38:00.768 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.170+0000 7f0e417fa700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0e2c038190 0x7f0e2c03a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:00.768 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.170+0000 7f0e417fa700 1 -- 172.21.15.32:0/63329804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0e3404c310 con 0x7f0e4407bea0 2023-08-11T15:38:00.768 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.170+0000 7f0e437fe700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0e2c038190 0x7f0e2c03a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:00.768 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.171+0000 7f0e437fe700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0e2c038190 0x7f0e2c03a650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e38006fd0 tx=0x7f0e38006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:00.768 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.175+0000 7f0e417fa700 1 -- 172.21.15.32:0/63329804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0e34010380 con 0x7f0e4407bea0 2023-08-11T15:38:00.768 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.388+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f0e44061f50 con 0x7f0e4407bea0 2023-08-11T15:38:00.768 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.389+0000 7f0e417fa700 1 -- 172.21.15.32:0/63329804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0e34010380 con 0x7f0e4407bea0 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0e2c038190 msgr2=0x7f0e2c03a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0e2c038190 0x7f0e2c03a650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e38006fd0 tx=0x7f0e38006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 msgr2=0x7f0e4419c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4419c050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0e34003e60 tx=0x7f0e34011a30 comp rx=0 tx=0).stop 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 shutdown_connections 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f0e2c038190 0x7f0e2c03a650 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 --2- 172.21.15.32:0/63329804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4407bea0 0x7f0e4419c050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:00.769 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 >> 172.21.15.32:0/63329804 conn(0x7f0e44101610 msgr2=0x7f0e441039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:00.770 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 shutdown_connections 2023-08-11T15:38:00.770 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:00.391+0000 7f0e4a446700 1 -- 172.21.15.32:0/63329804 wait complete. 2023-08-11T15:38:01.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:01 smithi032 ceph-mon[97103]: mgrmap e13: x(active, since 6s) 2023-08-11T15:38:01.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:01 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2305606136' entity='client.admin' 2023-08-11T15:38:02.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.236+0000 7fc9baebd700 1 Processor -- start 2023-08-11T15:38:02.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.236+0000 7fc9baebd700 1 -- start start 2023-08-11T15:38:02.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.236+0000 7fc9baebd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:02.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.236+0000 7fc9baebd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b407a8d0 con 0x7fc9b407bea0 2023-08-11T15:38:02.045 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b9ebb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b9ebb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48912/0 (socket says 172.21.15.32:48912) 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b9ebb700 1 -- 172.21.15.32:0/2024718819 learned_addr learned my addr 172.21.15.32:0/2024718819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b9ebb700 1 -- 172.21.15.32:0/2024718819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b407aa10 con 0x7fc9b407bea0 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b9ebb700 1 --2- 172.21.15.32:0/2024718819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b407a300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00098a0 tx=0x7fc9b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60778a820e703ac6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b8eb9700 1 -- 172.21.15.32:0/2024718819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b0004160 con 0x7fc9b407bea0 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b8eb9700 1 -- 172.21.15.32:0/2024718819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9b00042c0 con 0x7fc9b407bea0 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.237+0000 7fc9b8eb9700 1 -- 172.21.15.32:0/2024718819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b00119b0 con 0x7fc9b407bea0 2023-08-11T15:38:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.238+0000 7fc9baebd700 1 -- 172.21.15.32:0/2024718819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 msgr2=0x7fc9b407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.238+0000 7fc9baebd700 1 --2- 172.21.15.32:0/2024718819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b407a300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00098a0 tx=0x7fc9b0009bb0 comp rx=0 tx=0).stop 2023-08-11T15:38:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.238+0000 7fc9baebd700 1 -- 172.21.15.32:0/2024718819 shutdown_connections 2023-08-11T15:38:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.238+0000 7fc9baebd700 1 --2- 172.21.15.32:0/2024718819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b407a300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.238+0000 7fc9baebd700 1 -- 172.21.15.32:0/2024718819 >> 172.21.15.32:0/2024718819 conn(0x7fc9b4101440 msgr2=0x7fc9b4103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.238+0000 7fc9baebd700 1 -- 172.21.15.32:0/2024718819 shutdown_connections 2023-08-11T15:38:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.238+0000 7fc9baebd700 1 -- 172.21.15.32:0/2024718819 wait complete. 2023-08-11T15:38:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.239+0000 7fc9baebd700 1 Processor -- start 2023-08-11T15:38:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.239+0000 7fc9baebd700 1 -- start start 2023-08-11T15:38:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.240+0000 7fc9baebd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.240+0000 7fc9baebd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b419c3c0 con 0x7fc9b407bea0 2023-08-11T15:38:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.240+0000 7fc9b9ebb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.240+0000 7fc9b9ebb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48926/0 (socket says 172.21.15.32:48926) 2023-08-11T15:38:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.240+0000 7fc9b9ebb700 1 -- 172.21.15.32:0/2305606136 learned_addr learned my addr 172.21.15.32:0/2305606136 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:02.048 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.240+0000 7fc9b9ebb700 1 -- 172.21.15.32:0/2305606136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0009550 con 0x7fc9b407bea0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.241+0000 7fc9b9ebb700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b419be80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0003e60 tx=0x7fc9b0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.241+0000 7fc9aaffd700 1 -- 172.21.15.32:0/2305606136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b0003a60 con 0x7fc9b407bea0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.241+0000 7fc9aaffd700 1 -- 172.21.15.32:0/2305606136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9b0003bc0 con 0x7fc9b407bea0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.241+0000 7fc9aaffd700 1 -- 172.21.15.32:0/2305606136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b00196c0 con 0x7fc9b407bea0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.241+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b419c5c0 con 0x7fc9b407bea0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.241+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b419c9e0 con 0x7fc9b407bea0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.242+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b4195ea0 con 0x7fc9b407bea0 2023-08-11T15:38:02.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.242+0000 7fc9aaffd700 1 -- 172.21.15.32:0/2305606136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fc9b001ab60 con 0x7fc9b407bea0 2023-08-11T15:38:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.243+0000 7fc9aaffd700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc9a0038230 0x7fc9a003a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.243+0000 7fc9aaffd700 1 -- 172.21.15.32:0/2305606136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc9b004c340 con 0x7fc9b407bea0 2023-08-11T15:38:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.243+0000 7fc9b96ba700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc9a0038230 0x7fc9a003a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.243+0000 7fc9b96ba700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc9a0038230 0x7fc9a003a6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.247+0000 7fc9aaffd700 1 -- 172.21.15.32:0/2305606136 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc9b0010790 con 0x7fc9b407bea0 2023-08-11T15:38:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.566+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fc9b4061f50 con 0x7fc9b407bea0 2023-08-11T15:38:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.568+0000 7fc9aaffd700 1 -- 172.21.15.32:0/2305606136 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fc9b0018350 con 0x7fc9b407bea0 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc9a0038230 msgr2=0x7fc9a003a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc9a0038230 0x7fc9a003a6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 msgr2=0x7fc9b419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b419be80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0003e60 tx=0x7fc9b0011a30 comp rx=0 tx=0).stop 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 shutdown_connections 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc9a0038230 0x7fc9a003a6f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 --2- 172.21.15.32:0/2305606136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9b407bea0 0x7fc9b419be80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.570+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 >> 172.21.15.32:0/2305606136 conn(0x7fc9b4101440 msgr2=0x7fc9b4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:02.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.571+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 shutdown_connections 2023-08-11T15:38:02.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-08-11T15:38:01.571+0000 7fc9baebd700 1 -- 172.21.15.32:0/2305606136 wait complete. 2023-08-11T15:38:02.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-08-11T15:38:02.052 INFO:teuthology.orchestra.run.smithi032.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-08-11T15:38:02.052 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.052 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:02.052 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout:Or, if you are only running a single cluster on this host: 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout:Please consider enabling telemetry to help improve Ceph: 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout: ceph telemetry on 2023-08-11T15:38:02.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.054 INFO:teuthology.orchestra.run.smithi032.stdout:For more information see: 2023-08-11T15:38:02.054 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.054 INFO:teuthology.orchestra.run.smithi032.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-08-11T15:38:02.054 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:02.054 INFO:teuthology.orchestra.run.smithi032.stdout:Bootstrap complete. 2023-08-11T15:38:02.104 INFO:tasks.cephadm:Fetching config... 2023-08-11T15:38:02.104 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:38:02.105 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-08-11T15:38:02.148 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-08-11T15:38:02.148 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:38:02.149 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-08-11T15:38:02.232 INFO:tasks.cephadm:Fetching mon keyring... 2023-08-11T15:38:02.233 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:38:02.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-08-11T15:38:02.343 INFO:tasks.cephadm:Fetching pub ssh key... 2023-08-11T15:38:02.344 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:38:02.344 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-08-11T15:38:02.427 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-08-11T15:38:02.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCveyt56AAdBGlr5uIR9PbFU1HrjNTLGmYMOuBgMLhiin1hXL0hCm8Omig1nJsKaNAzqlcHP9NEweVcI3ayIFxYk8uD1bXLeAnJKZgMbQp1Aeby4ItPuKkorHk/elOneifVhCO276thYt51nBCQ+9G3/g+WUeZ9/waDr2BRt7lxLoqCJ7Pc6sD/8rD7RSzKd65eXZT415g5y8zNBQUrpKr/y7cnkmvE8HEAwRFEmwlyqYkUk2ioRAKOgW5g8H/GyaVtqAxdsZWCdWLOarVHwifLyxr7iJyP+WsZWTna9+r1gZzNe9ghKnvDVAl3o1F6VUcLuU8wzc9K9b7fHmnoHt/vtM/hy7ISr0cYqMBTpXdF1aI05KrRKPrk44qwvkXJ9LddOdC2hWfyVmefzFj4jyUINSNG07YNtebkcuSbzCsiNFI8hAzDeeOFOofyPDrCbr/rjz0GfyEh2zivc7uir3aTleeUWDNHA1Ab5dF+x2hUMnNi9OQJYglcSxVLNOtNFyM= ceph-cd36a838-385c-11ee-9b37-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-11T15:38:02.581 INFO:teuthology.orchestra.run.smithi032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCveyt56AAdBGlr5uIR9PbFU1HrjNTLGmYMOuBgMLhiin1hXL0hCm8Omig1nJsKaNAzqlcHP9NEweVcI3ayIFxYk8uD1bXLeAnJKZgMbQp1Aeby4ItPuKkorHk/elOneifVhCO276thYt51nBCQ+9G3/g+WUeZ9/waDr2BRt7lxLoqCJ7Pc6sD/8rD7RSzKd65eXZT415g5y8zNBQUrpKr/y7cnkmvE8HEAwRFEmwlyqYkUk2ioRAKOgW5g8H/GyaVtqAxdsZWCdWLOarVHwifLyxr7iJyP+WsZWTna9+r1gZzNe9ghKnvDVAl3o1F6VUcLuU8wzc9K9b7fHmnoHt/vtM/hy7ISr0cYqMBTpXdF1aI05KrRKPrk44qwvkXJ9LddOdC2hWfyVmefzFj4jyUINSNG07YNtebkcuSbzCsiNFI8hAzDeeOFOofyPDrCbr/rjz0GfyEh2zivc7uir3aTleeUWDNHA1Ab5dF+x2hUMnNi9OQJYglcSxVLNOtNFyM= ceph-cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:02.612 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCveyt56AAdBGlr5uIR9PbFU1HrjNTLGmYMOuBgMLhiin1hXL0hCm8Omig1nJsKaNAzqlcHP9NEweVcI3ayIFxYk8uD1bXLeAnJKZgMbQp1Aeby4ItPuKkorHk/elOneifVhCO276thYt51nBCQ+9G3/g+WUeZ9/waDr2BRt7lxLoqCJ7Pc6sD/8rD7RSzKd65eXZT415g5y8zNBQUrpKr/y7cnkmvE8HEAwRFEmwlyqYkUk2ioRAKOgW5g8H/GyaVtqAxdsZWCdWLOarVHwifLyxr7iJyP+WsZWTna9+r1gZzNe9ghKnvDVAl3o1F6VUcLuU8wzc9K9b7fHmnoHt/vtM/hy7ISr0cYqMBTpXdF1aI05KrRKPrk44qwvkXJ9LddOdC2hWfyVmefzFj4jyUINSNG07YNtebkcuSbzCsiNFI8hAzDeeOFOofyPDrCbr/rjz0GfyEh2zivc7uir3aTleeUWDNHA1Ab5dF+x2hUMnNi9OQJYglcSxVLNOtNFyM= ceph-cd36a838-385c-11ee-9b37-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-11T15:38:02.657 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCveyt56AAdBGlr5uIR9PbFU1HrjNTLGmYMOuBgMLhiin1hXL0hCm8Omig1nJsKaNAzqlcHP9NEweVcI3ayIFxYk8uD1bXLeAnJKZgMbQp1Aeby4ItPuKkorHk/elOneifVhCO276thYt51nBCQ+9G3/g+WUeZ9/waDr2BRt7lxLoqCJ7Pc6sD/8rD7RSzKd65eXZT415g5y8zNBQUrpKr/y7cnkmvE8HEAwRFEmwlyqYkUk2ioRAKOgW5g8H/GyaVtqAxdsZWCdWLOarVHwifLyxr7iJyP+WsZWTna9+r1gZzNe9ghKnvDVAl3o1F6VUcLuU8wzc9K9b7fHmnoHt/vtM/hy7ISr0cYqMBTpXdF1aI05KrRKPrk44qwvkXJ9LddOdC2hWfyVmefzFj4jyUINSNG07YNtebkcuSbzCsiNFI8hAzDeeOFOofyPDrCbr/rjz0GfyEh2zivc7uir3aTleeUWDNHA1Ab5dF+x2hUMnNi9OQJYglcSxVLNOtNFyM= ceph-cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:02.670 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCveyt56AAdBGlr5uIR9PbFU1HrjNTLGmYMOuBgMLhiin1hXL0hCm8Omig1nJsKaNAzqlcHP9NEweVcI3ayIFxYk8uD1bXLeAnJKZgMbQp1Aeby4ItPuKkorHk/elOneifVhCO276thYt51nBCQ+9G3/g+WUeZ9/waDr2BRt7lxLoqCJ7Pc6sD/8rD7RSzKd65eXZT415g5y8zNBQUrpKr/y7cnkmvE8HEAwRFEmwlyqYkUk2ioRAKOgW5g8H/GyaVtqAxdsZWCdWLOarVHwifLyxr7iJyP+WsZWTna9+r1gZzNe9ghKnvDVAl3o1F6VUcLuU8wzc9K9b7fHmnoHt/vtM/hy7ISr0cYqMBTpXdF1aI05KrRKPrk44qwvkXJ9LddOdC2hWfyVmefzFj4jyUINSNG07YNtebkcuSbzCsiNFI8hAzDeeOFOofyPDrCbr/rjz0GfyEh2zivc7uir3aTleeUWDNHA1Ab5dF+x2hUMnNi9OQJYglcSxVLNOtNFyM= ceph-cd36a838-385c-11ee-9b37-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-11T15:38:02.715 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCveyt56AAdBGlr5uIR9PbFU1HrjNTLGmYMOuBgMLhiin1hXL0hCm8Omig1nJsKaNAzqlcHP9NEweVcI3ayIFxYk8uD1bXLeAnJKZgMbQp1Aeby4ItPuKkorHk/elOneifVhCO276thYt51nBCQ+9G3/g+WUeZ9/waDr2BRt7lxLoqCJ7Pc6sD/8rD7RSzKd65eXZT415g5y8zNBQUrpKr/y7cnkmvE8HEAwRFEmwlyqYkUk2ioRAKOgW5g8H/GyaVtqAxdsZWCdWLOarVHwifLyxr7iJyP+WsZWTna9+r1gZzNe9ghKnvDVAl3o1F6VUcLuU8wzc9K9b7fHmnoHt/vtM/hy7ISr0cYqMBTpXdF1aI05KrRKPrk44qwvkXJ9LddOdC2hWfyVmefzFj4jyUINSNG07YNtebkcuSbzCsiNFI8hAzDeeOFOofyPDrCbr/rjz0GfyEh2zivc7uir3aTleeUWDNHA1Ab5dF+x2hUMnNi9OQJYglcSxVLNOtNFyM= ceph-cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:02.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-08-11T15:38:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:05.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:05.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:05.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:38:05.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:05.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:05.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:05.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.105+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3873840874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 msgr2=0x7fdb48100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:05.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.105+0000 7fdb4cdd1700 1 --2- 172.21.15.32:0/3873840874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48100850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009910 tx=0x7fdb38009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:05.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.105+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3873840874 shutdown_connections 2023-08-11T15:38:05.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.105+0000 7fdb4cdd1700 1 --2- 172.21.15.32:0/3873840874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48100850 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:05.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.105+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3873840874 >> 172.21.15.32:0/3873840874 conn(0x7fdb480fb9b0 msgr2=0x7fdb480fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:05.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.105+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3873840874 shutdown_connections 2023-08-11T15:38:05.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.105+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3873840874 wait complete. 2023-08-11T15:38:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.106+0000 7fdb4cdd1700 1 Processor -- start 2023-08-11T15:38:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.106+0000 7fdb4cdd1700 1 -- start start 2023-08-11T15:38:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.107+0000 7fdb4cdd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:05.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.107+0000 7fdb4cdd1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48197c90 con 0x7fdb48100430 2023-08-11T15:38:05.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.107+0000 7fdb4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:05.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.107+0000 7fdb4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44752/0 (socket says 172.21.15.32:44752) 2023-08-11T15:38:05.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.107+0000 7fdb4659c700 1 -- 172.21.15.32:0/3621122450 learned_addr learned my addr 172.21.15.32:0/3621122450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:05.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.107+0000 7fdb4659c700 1 -- 172.21.15.32:0/3621122450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380095f0 con 0x7fdb48100430 2023-08-11T15:38:05.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.108+0000 7fdb4659c700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48197750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdb38006010 tx=0x7fdb380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:05.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.108+0000 7fdb377fe700 1 -- 172.21.15.32:0/3621122450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb3801d070 con 0x7fdb48100430 2023-08-11T15:38:05.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.108+0000 7fdb377fe700 1 -- 172.21.15.32:0/3621122450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb380044e0 con 0x7fdb48100430 2023-08-11T15:38:05.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.108+0000 7fdb377fe700 1 -- 172.21.15.32:0/3621122450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb380039b0 con 0x7fdb48100430 2023-08-11T15:38:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.108+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb48197e90 con 0x7fdb48100430 2023-08-11T15:38:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.108+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb481982d0 con 0x7fdb48100430 2023-08-11T15:38:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.109+0000 7fdb377fe700 1 -- 172.21.15.32:0/3621122450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fdb3800baf0 con 0x7fdb48100430 2023-08-11T15:38:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.109+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb48191340 con 0x7fdb48100430 2023-08-11T15:38:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.109+0000 7fdb377fe700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdb30038200 0x7fdb3003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.110+0000 7fdb377fe700 1 -- 172.21.15.32:0/3621122450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb3804c1b0 con 0x7fdb48100430 2023-08-11T15:38:05.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.110+0000 7fdb45d9b700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdb30038200 0x7fdb3003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:05.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.110+0000 7fdb45d9b700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdb30038200 0x7fdb3003a6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006fd0 tx=0x7fdb3c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:05.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.115+0000 7fdb377fe700 1 -- 172.21.15.32:0/3621122450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fdb3802a950 con 0x7fdb48100430 2023-08-11T15:38:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.320+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fdb48106180 con 0x7fdb48100430 2023-08-11T15:38:05.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.324+0000 7fdb377fe700 1 -- 172.21.15.32:0/3621122450 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fdb3802a360 con 0x7fdb48100430 2023-08-11T15:38:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.326+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdb30038200 msgr2=0x7fdb3003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.326+0000 7fdb4cdd1700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdb30038200 0x7fdb3003a6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006fd0 tx=0x7fdb3c006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.326+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 msgr2=0x7fdb48197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.326+0000 7fdb4cdd1700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48197750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdb38006010 tx=0x7fdb380057c0 comp rx=0 tx=0).stop 2023-08-11T15:38:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.327+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 shutdown_connections 2023-08-11T15:38:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.327+0000 7fdb4cdd1700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdb30038200 0x7fdb3003a6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.327+0000 7fdb4cdd1700 1 --2- 172.21.15.32:0/3621122450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb48100430 0x7fdb48197750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.327+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 >> 172.21.15.32:0/3621122450 conn(0x7fdb480fb9b0 msgr2=0x7fdb480fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.327+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 shutdown_connections 2023-08-11T15:38:05.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:05.327+0000 7fdb4cdd1700 1 -- 172.21.15.32:0/3621122450 wait complete. 2023-08-11T15:38:05.907 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-08-11T15:38:05.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-08-11T15:38:06.638 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:06.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:06 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3621122450' entity='client.admin' 2023-08-11T15:38:06.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:06.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.024+0000 7f413e60f700 1 -- 172.21.15.32:0/2987015237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 msgr2=0x7f4138100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.024+0000 7f413e60f700 1 --2- 172.21.15.32:0/2987015237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f4138100850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4128009910 tx=0x7f4128009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.024+0000 7f413e60f700 1 -- 172.21.15.32:0/2987015237 shutdown_connections 2023-08-11T15:38:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.024+0000 7f413e60f700 1 --2- 172.21.15.32:0/2987015237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f4138100850 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:08.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.024+0000 7f413e60f700 1 -- 172.21.15.32:0/2987015237 >> 172.21.15.32:0/2987015237 conn(0x7f41380fb9b0 msgr2=0x7f41380fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.024+0000 7f413e60f700 1 -- 172.21.15.32:0/2987015237 shutdown_connections 2023-08-11T15:38:08.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.024+0000 7f413e60f700 1 -- 172.21.15.32:0/2987015237 wait complete. 2023-08-11T15:38:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.025+0000 7f413e60f700 1 Processor -- start 2023-08-11T15:38:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.025+0000 7f413e60f700 1 -- start start 2023-08-11T15:38:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.026+0000 7f413e60f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f41380746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.026+0000 7f413e60f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138074c00 con 0x7f4138100430 2023-08-11T15:38:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.026+0000 7f4137fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f41380746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:08.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.026+0000 7f4137fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f41380746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44758/0 (socket says 172.21.15.32:44758) 2023-08-11T15:38:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.026+0000 7f4137fff700 1 -- 172.21.15.32:0/4244252380 learned_addr learned my addr 172.21.15.32:0/4244252380 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.026+0000 7f4137fff700 1 -- 172.21.15.32:0/4244252380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41280095f0 con 0x7f4138100430 2023-08-11T15:38:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.026+0000 7f4137fff700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f41380746c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4128004f40 tx=0x7f41280057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.027+0000 7f41357fa700 1 -- 172.21.15.32:0/4244252380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f412801d070 con 0x7f4138100430 2023-08-11T15:38:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.027+0000 7f41357fa700 1 -- 172.21.15.32:0/4244252380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f412800bc10 con 0x7f4138100430 2023-08-11T15:38:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.027+0000 7f41357fa700 1 -- 172.21.15.32:0/4244252380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4128003a70 con 0x7f4138100430 2023-08-11T15:38:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.027+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4138072d10 con 0x7f4138100430 2023-08-11T15:38:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.027+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41380731b0 con 0x7f4138100430 2023-08-11T15:38:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.028+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138061f50 con 0x7f4138100430 2023-08-11T15:38:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.028+0000 7f41357fa700 1 -- 172.21.15.32:0/4244252380 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f4128003ce0 con 0x7f4138100430 2023-08-11T15:38:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.028+0000 7f41357fa700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4120038200 0x7f412003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.028+0000 7f41357fa700 1 -- 172.21.15.32:0/4244252380 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f412804bdf0 con 0x7f4138100430 2023-08-11T15:38:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.029+0000 7f41377fe700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4120038200 0x7f412003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.029+0000 7f41377fe700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4120038200 0x7f412003a6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f412c006fd0 tx=0x7f412c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:08.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.033+0000 7f41357fa700 1 -- 172.21.15.32:0/4244252380 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f412802abf0 con 0x7f4138100430 2023-08-11T15:38:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.245+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f413818ec50 con 0x7f4120038200 2023-08-11T15:38:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.248+0000 7f41357fa700 1 -- 172.21.15.32:0/4244252380 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f413818ec50 con 0x7f4120038200 2023-08-11T15:38:08.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4120038200 msgr2=0x7f412003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:08.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4120038200 0x7f412003a6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f412c006fd0 tx=0x7f412c006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:08.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 msgr2=0x7f41380746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:08.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f41380746c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4128004f40 tx=0x7f41280057c0 comp rx=0 tx=0).stop 2023-08-11T15:38:08.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 shutdown_connections 2023-08-11T15:38:08.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4120038200 0x7f412003a6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:08.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 --2- 172.21.15.32:0/4244252380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4138100430 0x7f41380746c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:08.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 >> 172.21.15.32:0/4244252380 conn(0x7f41380fb9b0 msgr2=0x7f41380fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:08.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 shutdown_connections 2023-08-11T15:38:08.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:08.251+0000 7f413e60f700 1 -- 172.21.15.32:0/4244252380 wait complete. 2023-08-11T15:38:08.720 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2023-08-11T15:38:08.720 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:38:08.721 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2023-08-11T15:38:08.742 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:38:08.742 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:08.802 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2023-08-11T15:38:08.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch host add smithi134 2023-08-11T15:38:09.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:09 smithi032 ceph-mon[97103]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:09.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:09 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:09.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:09 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:09.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:09 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:09.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:09 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:09.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:09 smithi032 ceph-mon[97103]: Updating smithi032:/etc/ceph/ceph.conf 2023-08-11T15:38:09.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:09 smithi032 ceph-mon[97103]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:10.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:10 smithi032 ceph-mon[97103]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:10.458 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:10 smithi032 ceph-mon[97103]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:38:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.052+0000 7f6f1566f700 1 -- 172.21.15.32:0/982993866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 msgr2=0x7f6f101009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.052+0000 7f6f1566f700 1 --2- 172.21.15.32:0/982993866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f101009f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009910 tx=0x7f6f00009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.053+0000 7f6f1566f700 1 -- 172.21.15.32:0/982993866 shutdown_connections 2023-08-11T15:38:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.053+0000 7f6f1566f700 1 --2- 172.21.15.32:0/982993866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f101009f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.053+0000 7f6f1566f700 1 -- 172.21.15.32:0/982993866 >> 172.21.15.32:0/982993866 conn(0x7f6f100fbee0 msgr2=0x7f6f100fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.053+0000 7f6f1566f700 1 -- 172.21.15.32:0/982993866 shutdown_connections 2023-08-11T15:38:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.053+0000 7f6f1566f700 1 -- 172.21.15.32:0/982993866 wait complete. 2023-08-11T15:38:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.054+0000 7f6f1566f700 1 Processor -- start 2023-08-11T15:38:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.054+0000 7f6f1566f700 1 -- start start 2023-08-11T15:38:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.054+0000 7f6f1566f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f1019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.054+0000 7f6f1566f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1019ddd0 con 0x7f6f10100610 2023-08-11T15:38:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6f0effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f1019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6f0effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f1019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44772/0 (socket says 172.21.15.32:44772) 2023-08-11T15:38:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6f0effd700 1 -- 172.21.15.32:0/694626512 learned_addr learned my addr 172.21.15.32:0/694626512 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6f0effd700 1 -- 172.21.15.32:0/694626512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f000095f0 con 0x7f6f10100610 2023-08-11T15:38:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6f0effd700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f1019d890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f00006010 tx=0x7f6f00004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6ef7fff700 1 -- 172.21.15.32:0/694626512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f0001d070 con 0x7f6f10100610 2023-08-11T15:38:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6ef7fff700 1 -- 172.21.15.32:0/694626512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f0000bc10 con 0x7f6f10100610 2023-08-11T15:38:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6ef7fff700 1 -- 172.21.15.32:0/694626512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f00003910 con 0x7f6f10100610 2023-08-11T15:38:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.055+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1019dfd0 con 0x7f6f10100610 2023-08-11T15:38:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.056+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1019e3f0 con 0x7f6f10100610 2023-08-11T15:38:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.056+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f10061f50 con 0x7f6f10100610 2023-08-11T15:38:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.057+0000 7f6ef7fff700 1 -- 172.21.15.32:0/694626512 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f6f0000bd80 con 0x7f6f10100610 2023-08-11T15:38:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.057+0000 7f6ef7fff700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6ef8038200 0x7f6ef803a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.057+0000 7f6ef7fff700 1 -- 172.21.15.32:0/694626512 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6f0004bdf0 con 0x7f6f10100610 2023-08-11T15:38:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.057+0000 7f6f0e7fc700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6ef8038200 0x7f6ef803a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.058+0000 7f6f0e7fc700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6ef8038200 0x7f6ef803a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f04006fd0 tx=0x7f6f04006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:11.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.062+0000 7f6ef7fff700 1 -- 172.21.15.32:0/694626512 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6f00029360 con 0x7f6f10100610 2023-08-11T15:38:11.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:11.275+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f1010a920 con 0x7f6ef8038200 2023-08-11T15:38:11.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:11 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:11.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:11 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:11.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:11 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:13.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:12 smithi032 ceph-mon[97103]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:13.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:12 smithi032 ceph-mon[97103]: Deploying cephadm binary to smithi134 2023-08-11T15:38:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.631+0000 7f6ef7fff700 1 -- 172.21.15.32:0/694626512 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6f1010a920 con 0x7f6ef8038200 2023-08-11T15:38:13.634 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi134' with addr '172.21.15.134' 2023-08-11T15:38:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6ef8038200 msgr2=0x7f6ef803a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6ef8038200 0x7f6ef803a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f04006fd0 tx=0x7f6f04006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 msgr2=0x7f6f1019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f1019d890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f00006010 tx=0x7f6f00004ff0 comp rx=0 tx=0).stop 2023-08-11T15:38:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 shutdown_connections 2023-08-11T15:38:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6ef8038200 0x7f6ef803a6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 --2- 172.21.15.32:0/694626512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10100610 0x7f6f1019d890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 >> 172.21.15.32:0/694626512 conn(0x7f6f100fbee0 msgr2=0x7f6f10103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 shutdown_connections 2023-08-11T15:38:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:13.634+0000 7f6f1566f700 1 -- 172.21.15.32:0/694626512 wait complete. 2023-08-11T15:38:14.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch host ls --format=json 2023-08-11T15:38:15.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:15.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:14 smithi032 ceph-mon[97103]: Added host smithi134 2023-08-11T15:38:15.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:15.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:14 smithi032 ceph-mon[97103]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:15.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:15.082 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:15.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:15.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.467+0000 7ffa13fba700 1 -- 172.21.15.32:0/3962557629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 msgr2=0x7ffa0c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.467+0000 7ffa13fba700 1 --2- 172.21.15.32:0/3962557629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c108a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009910 tx=0x7ffa08009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.467+0000 7ffa13fba700 1 -- 172.21.15.32:0/3962557629 shutdown_connections 2023-08-11T15:38:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.467+0000 7ffa13fba700 1 --2- 172.21.15.32:0/3962557629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c108a50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.467+0000 7ffa13fba700 1 -- 172.21.15.32:0/3962557629 >> 172.21.15.32:0/3962557629 conn(0x7ffa0c0fe0d0 msgr2=0x7ffa0c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.467+0000 7ffa13fba700 1 -- 172.21.15.32:0/3962557629 shutdown_connections 2023-08-11T15:38:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.467+0000 7ffa13fba700 1 -- 172.21.15.32:0/3962557629 wait complete. 2023-08-11T15:38:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.468+0000 7ffa13fba700 1 Processor -- start 2023-08-11T15:38:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.468+0000 7ffa13fba700 1 -- start start 2023-08-11T15:38:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.468+0000 7ffa13fba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.468+0000 7ffa13fba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c19dc10 con 0x7ffa0c108670 2023-08-11T15:38:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.469+0000 7ffa11d56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.469+0000 7ffa11d56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53588/0 (socket says 172.21.15.32:53588) 2023-08-11T15:38:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.469+0000 7ffa11d56700 1 -- 172.21.15.32:0/4197279673 learned_addr learned my addr 172.21.15.32:0/4197279673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.469+0000 7ffa11d56700 1 -- 172.21.15.32:0/4197279673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080095f0 con 0x7ffa0c108670 2023-08-11T15:38:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.469+0000 7ffa11d56700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c19d6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa08006010 tx=0x7ffa08004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.470+0000 7ffa02ffd700 1 -- 172.21.15.32:0/4197279673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa0801d070 con 0x7ffa0c108670 2023-08-11T15:38:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.470+0000 7ffa02ffd700 1 -- 172.21.15.32:0/4197279673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0800bc10 con 0x7ffa0c108670 2023-08-11T15:38:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.470+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c19de70 con 0x7ffa0c108670 2023-08-11T15:38:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.470+0000 7ffa02ffd700 1 -- 172.21.15.32:0/4197279673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa08003910 con 0x7ffa0c108670 2023-08-11T15:38:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.470+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c197770 con 0x7ffa0c108670 2023-08-11T15:38:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.471+0000 7ffa02ffd700 1 -- 172.21.15.32:0/4197279673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7ffa0800bd80 con 0x7ffa0c108670 2023-08-11T15:38:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.471+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c061f50 con 0x7ffa0c108670 2023-08-11T15:38:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.471+0000 7ffa02ffd700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff9f8038200 0x7ff9f803a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.471+0000 7ffa02ffd700 1 -- 172.21.15.32:0/4197279673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffa0804bdc0 con 0x7ffa0c108670 2023-08-11T15:38:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.472+0000 7ffa11555700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff9f8038200 0x7ff9f803a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.472+0000 7ffa11555700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff9f8038200 0x7ff9f803a6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006fd0 tx=0x7ff9fc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.477+0000 7ffa02ffd700 1 -- 172.21.15.32:0/4197279673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ffa0802ad00 con 0x7ffa0c108670 2023-08-11T15:38:16.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.693+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa0c10c950 con 0x7ff9f8038200 2023-08-11T15:38:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.695+0000 7ffa02ffd700 1 -- 172.21.15.32:0/4197279673 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7ffa0c10c950 con 0x7ff9f8038200 2023-08-11T15:38:16.697 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:16.697 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2023-08-11T15:38:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff9f8038200 msgr2=0x7ff9f803a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:16.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff9f8038200 0x7ff9f803a6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006fd0 tx=0x7ff9fc006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:16.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 msgr2=0x7ffa0c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:16.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c19d6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa08006010 tx=0x7ffa08004ff0 comp rx=0 tx=0).stop 2023-08-11T15:38:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 shutdown_connections 2023-08-11T15:38:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff9f8038200 0x7ff9f803a6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 --2- 172.21.15.32:0/4197279673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa0c108670 0x7ffa0c19d6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:16.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 >> 172.21.15.32:0/4197279673 conn(0x7ffa0c0fe0d0 msgr2=0x7ffa0c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 shutdown_connections 2023-08-11T15:38:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:16.697+0000 7ffa13fba700 1 -- 172.21.15.32:0/4197279673 wait complete. 2023-08-11T15:38:16.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:16 smithi032 ceph-mon[97103]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:17.260 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2023-08-11T15:38:17.260 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:38:17.261 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2023-08-11T15:38:17.282 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:38:17.282 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:17.342 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2023-08-11T15:38:17.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch host add smithi192 2023-08-11T15:38:18.052 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:17 smithi032 ceph-mon[97103]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:38:18.073 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:19.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:19.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:38:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: Updating smithi134:/etc/ceph/ceph.conf 2023-08-11T15:38:19.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:19.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:19.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:19 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:19.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.602+0000 7faba88a8700 1 -- 172.21.15.32:0/3577340696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 msgr2=0x7faba41009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.602+0000 7faba88a8700 1 --2- 172.21.15.32:0/3577340696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba41009f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab94009910 tx=0x7fab94009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.602+0000 7faba88a8700 1 -- 172.21.15.32:0/3577340696 shutdown_connections 2023-08-11T15:38:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.602+0000 7faba88a8700 1 --2- 172.21.15.32:0/3577340696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba41009f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.602+0000 7faba88a8700 1 -- 172.21.15.32:0/3577340696 >> 172.21.15.32:0/3577340696 conn(0x7faba40fbee0 msgr2=0x7faba40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.602+0000 7faba88a8700 1 -- 172.21.15.32:0/3577340696 shutdown_connections 2023-08-11T15:38:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.602+0000 7faba88a8700 1 -- 172.21.15.32:0/3577340696 wait complete. 2023-08-11T15:38:19.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.604+0000 7faba88a8700 1 Processor -- start 2023-08-11T15:38:19.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.604+0000 7faba88a8700 1 -- start start 2023-08-11T15:38:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.604+0000 7faba88a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.604+0000 7faba88a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba419baf0 con 0x7faba4100610 2023-08-11T15:38:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.605+0000 7faba259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.605+0000 7faba259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba419b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53608/0 (socket says 172.21.15.32:53608) 2023-08-11T15:38:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.605+0000 7faba259c700 1 -- 172.21.15.32:0/242234576 learned_addr learned my addr 172.21.15.32:0/242234576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.605+0000 7faba259c700 1 -- 172.21.15.32:0/242234576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab940095f0 con 0x7faba4100610 2023-08-11T15:38:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.605+0000 7faba259c700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba419b5b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab94004f40 tx=0x7fab94005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.605+0000 7fab937fe700 1 -- 172.21.15.32:0/242234576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab9401d070 con 0x7faba4100610 2023-08-11T15:38:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.606+0000 7fab937fe700 1 -- 172.21.15.32:0/242234576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab94004370 con 0x7faba4100610 2023-08-11T15:38:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.606+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba419bcf0 con 0x7faba4100610 2023-08-11T15:38:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.606+0000 7fab937fe700 1 -- 172.21.15.32:0/242234576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab940039b0 con 0x7faba4100610 2023-08-11T15:38:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.606+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba419c110 con 0x7faba4100610 2023-08-11T15:38:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.607+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba4061f50 con 0x7faba4100610 2023-08-11T15:38:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.607+0000 7fab937fe700 1 -- 172.21.15.32:0/242234576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fab94003c20 con 0x7faba4100610 2023-08-11T15:38:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.607+0000 7fab937fe700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab8c038200 0x7fab8c03a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.607+0000 7fab937fe700 1 -- 172.21.15.32:0/242234576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fab9404bfe0 con 0x7faba4100610 2023-08-11T15:38:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.607+0000 7faba1d9b700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab8c038200 0x7fab8c03a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.608+0000 7faba1d9b700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab8c038200 0x7fab8c03a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fab98006fd0 tx=0x7fab98006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:19.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.612+0000 7fab937fe700 1 -- 172.21.15.32:0/242234576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fab94029360 con 0x7faba4100610 2023-08-11T15:38:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:19.824+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7faba410a910 con 0x7fab8c038200 2023-08-11T15:38:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:20 smithi032 ceph-mon[97103]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:20 smithi032 ceph-mon[97103]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:20.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:20 smithi032 ceph-mon[97103]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:38:21.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:21 smithi032 ceph-mon[97103]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:21.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:21 smithi032 ceph-mon[97103]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.221+0000 7fab937fe700 1 -- 172.21.15.32:0/242234576 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7faba410a910 con 0x7fab8c038200 2023-08-11T15:38:22.224 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi192' with addr '172.21.15.192' 2023-08-11T15:38:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.223+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab8c038200 msgr2=0x7fab8c03a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.223+0000 7faba88a8700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab8c038200 0x7fab8c03a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fab98006fd0 tx=0x7fab98006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 msgr2=0x7faba419b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:22.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba419b5b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab94004f40 tx=0x7fab94005020 comp rx=0 tx=0).stop 2023-08-11T15:38:22.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 shutdown_connections 2023-08-11T15:38:22.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab8c038200 0x7fab8c03a6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:22.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 --2- 172.21.15.32:0/242234576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faba4100610 0x7faba419b5b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:22.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 >> 172.21.15.32:0/242234576 conn(0x7faba40fbee0 msgr2=0x7faba4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:22.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 shutdown_connections 2023-08-11T15:38:22.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:22.224+0000 7faba88a8700 1 -- 172.21.15.32:0/242234576 wait complete. 2023-08-11T15:38:22.440 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:22 smithi032 ceph-mon[97103]: Deploying cephadm binary to smithi192 2023-08-11T15:38:22.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch host ls --format=json 2023-08-11T15:38:23.402 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:23.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:23 smithi032 ceph-mon[97103]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:23.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:23.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:23 smithi032 ceph-mon[97103]: Added host smithi192 2023-08-11T15:38:23.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:23.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:24.552 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:24 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:24.552 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:24 smithi032 ceph-mon[97103]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.791+0000 7f6ca476f700 1 -- 172.21.15.32:0/1453048509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 msgr2=0x7f6c9c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.791+0000 7f6ca476f700 1 --2- 172.21.15.32:0/1453048509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c1009f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009910 tx=0x7f6c98009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.792+0000 7f6ca476f700 1 -- 172.21.15.32:0/1453048509 shutdown_connections 2023-08-11T15:38:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.792+0000 7f6ca476f700 1 --2- 172.21.15.32:0/1453048509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c1009f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.792+0000 7f6ca476f700 1 -- 172.21.15.32:0/1453048509 >> 172.21.15.32:0/1453048509 conn(0x7f6c9c0fbee0 msgr2=0x7f6c9c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.792+0000 7f6ca476f700 1 -- 172.21.15.32:0/1453048509 shutdown_connections 2023-08-11T15:38:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.792+0000 7f6ca476f700 1 -- 172.21.15.32:0/1453048509 wait complete. 2023-08-11T15:38:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.793+0000 7f6ca476f700 1 Processor -- start 2023-08-11T15:38:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.793+0000 7f6ca476f700 1 -- start start 2023-08-11T15:38:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.794+0000 7f6ca476f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.794+0000 7f6ca476f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c19de90 con 0x7f6c9c100610 2023-08-11T15:38:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.794+0000 7f6ca250b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.794+0000 7f6ca250b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49526/0 (socket says 172.21.15.32:49526) 2023-08-11T15:38:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.794+0000 7f6ca250b700 1 -- 172.21.15.32:0/492811269 learned_addr learned my addr 172.21.15.32:0/492811269 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.795+0000 7f6ca250b700 1 -- 172.21.15.32:0/492811269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980095f0 con 0x7f6c9c100610 2023-08-11T15:38:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.795+0000 7f6ca250b700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c19d950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c98004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.795+0000 7f6c937fe700 1 -- 172.21.15.32:0/492811269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c9801d070 con 0x7f6c9c100610 2023-08-11T15:38:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.795+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c19e090 con 0x7f6c9c100610 2023-08-11T15:38:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.795+0000 7f6c937fe700 1 -- 172.21.15.32:0/492811269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9800bc10 con 0x7f6c9c100610 2023-08-11T15:38:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.795+0000 7f6c937fe700 1 -- 172.21.15.32:0/492811269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c98003910 con 0x7f6c9c100610 2023-08-11T15:38:24.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.795+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c197940 con 0x7f6c9c100610 2023-08-11T15:38:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.796+0000 7f6c937fe700 1 -- 172.21.15.32:0/492811269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f6c9800bd80 con 0x7f6c9c100610 2023-08-11T15:38:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.796+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c19e4b0 con 0x7f6c9c100610 2023-08-11T15:38:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.797+0000 7f6c937fe700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c880381b0 0x7f6c8803a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.797+0000 7f6c937fe700 1 -- 172.21.15.32:0/492811269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6c9804be20 con 0x7f6c9c100610 2023-08-11T15:38:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.797+0000 7f6ca1d0a700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c880381b0 0x7f6c8803a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.798+0000 7f6ca1d0a700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c880381b0 0x7f6c8803a670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c006fd0 tx=0x7f6c8c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:24.802+0000 7f6c937fe700 1 -- 172.21.15.32:0/492811269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6c98029360 con 0x7f6c9c100610 2023-08-11T15:38:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.018+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c9c10a770 con 0x7f6c880381b0 2023-08-11T15:38:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.019+0000 7f6c937fe700 1 -- 172.21.15.32:0/492811269 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f6c9c10a770 con 0x7f6c880381b0 2023-08-11T15:38:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:38:25.022 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2023-08-11T15:38:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c880381b0 msgr2=0x7f6c8803a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c880381b0 0x7f6c8803a670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c006fd0 tx=0x7f6c8c006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 msgr2=0x7f6c9c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c19d950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c98004ff0 comp rx=0 tx=0).stop 2023-08-11T15:38:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 shutdown_connections 2023-08-11T15:38:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c880381b0 0x7f6c8803a670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 --2- 172.21.15.32:0/492811269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9c100610 0x7f6c9c19d950 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 >> 172.21.15.32:0/492811269 conn(0x7f6c9c0fbee0 msgr2=0x7f6c9c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 shutdown_connections 2023-08-11T15:38:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:25.022+0000 7f6ca476f700 1 -- 172.21.15.32:0/492811269 wait complete. 2023-08-11T15:38:25.483 INFO:tasks.cephadm:Setting crush tunables to default 2023-08-11T15:38:25.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd crush tunables default 2023-08-11T15:38:25.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:25 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:25.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:25 smithi032 ceph-mon[97103]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:38:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:26.787 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:26 smithi032 ceph-mon[97103]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.730+0000 7fd1ccf18700 1 -- 172.21.15.32:0/3883743677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 msgr2=0x7fd1c8100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.730+0000 7fd1ccf18700 1 --2- 172.21.15.32:0/3883743677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c8100970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009910 tx=0x7fd1b8009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.730+0000 7fd1ccf18700 1 -- 172.21.15.32:0/3883743677 shutdown_connections 2023-08-11T15:38:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.730+0000 7fd1ccf18700 1 --2- 172.21.15.32:0/3883743677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c8100970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.730+0000 7fd1ccf18700 1 -- 172.21.15.32:0/3883743677 >> 172.21.15.32:0/3883743677 conn(0x7fd1c80fbe80 msgr2=0x7fd1c80fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.730+0000 7fd1ccf18700 1 -- 172.21.15.32:0/3883743677 shutdown_connections 2023-08-11T15:38:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.730+0000 7fd1ccf18700 1 -- 172.21.15.32:0/3883743677 wait complete. 2023-08-11T15:38:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.731+0000 7fd1ccf18700 1 Processor -- start 2023-08-11T15:38:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.731+0000 7fd1ccf18700 1 -- start start 2023-08-11T15:38:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.732+0000 7fd1ccf18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.732+0000 7fd1ccf18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c819bb30 con 0x7fd1c8100590 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.732+0000 7fd1c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.732+0000 7fd1c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49546/0 (socket says 172.21.15.32:49546) 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.732+0000 7fd1c659c700 1 -- 172.21.15.32:0/2899386737 learned_addr learned my addr 172.21.15.32:0/2899386737 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.732+0000 7fd1c659c700 1 -- 172.21.15.32:0/2899386737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b80095f0 con 0x7fd1c8100590 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.733+0000 7fd1c659c700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c819b5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.733+0000 7fd1b77fe700 1 -- 172.21.15.32:0/2899386737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1b801d070 con 0x7fd1c8100590 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.733+0000 7fd1b77fe700 1 -- 172.21.15.32:0/2899386737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1b800bc10 con 0x7fd1c8100590 2023-08-11T15:38:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.733+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c819bd30 con 0x7fd1c8100590 2023-08-11T15:38:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.733+0000 7fd1b77fe700 1 -- 172.21.15.32:0/2899386737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1b8003910 con 0x7fd1c8100590 2023-08-11T15:38:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.733+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c819c150 con 0x7fd1c8100590 2023-08-11T15:38:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.734+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c8061f50 con 0x7fd1c8100590 2023-08-11T15:38:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.734+0000 7fd1b77fe700 1 -- 172.21.15.32:0/2899386737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fd1b800bd80 con 0x7fd1c8100590 2023-08-11T15:38:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.735+0000 7fd1b77fe700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1b0038200 0x7fd1b003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.735+0000 7fd1b77fe700 1 -- 172.21.15.32:0/2899386737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd1b804bdf0 con 0x7fd1c8100590 2023-08-11T15:38:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.735+0000 7fd1c5d9b700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1b0038200 0x7fd1b003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.735+0000 7fd1c5d9b700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1b0038200 0x7fd1b003a6c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc006fd0 tx=0x7fd1bc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.740+0000 7fd1b77fe700 1 -- 172.21.15.32:0/2899386737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd1b8029360 con 0x7fd1c8100590 2023-08-11T15:38:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:27.954+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd1c810a7a0 con 0x7fd1c8100590 2023-08-11T15:38:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.167+0000 7fd1b77fe700 1 -- 172.21.15.32:0/2899386737 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd1b8026020 con 0x7fd1c8100590 2023-08-11T15:38:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.169+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1b0038200 msgr2=0x7fd1b003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.169+0000 7fd1ccf18700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1b0038200 0x7fd1b003a6c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc006fd0 tx=0x7fd1bc006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.169+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 msgr2=0x7fd1c819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.169+0000 7fd1ccf18700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c819b5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b8004ff0 comp rx=0 tx=0).stop 2023-08-11T15:38:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.170+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 shutdown_connections 2023-08-11T15:38:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.170+0000 7fd1ccf18700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1b0038200 0x7fd1b003a6c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.170+0000 7fd1ccf18700 1 --2- 172.21.15.32:0/2899386737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1c8100590 0x7fd1c819b5f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.170+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 >> 172.21.15.32:0/2899386737 conn(0x7fd1c80fbe80 msgr2=0x7fd1c8104860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.170+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 shutdown_connections 2023-08-11T15:38:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:28.170+0000 7fd1ccf18700 1 -- 172.21.15.32:0/2899386737 wait complete. 2023-08-11T15:38:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2023-08-11T15:38:28.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:38:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:38:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2899386737' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-11T15:38:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:28.599 INFO:tasks.cephadm:Adding mon.a on smithi032 2023-08-11T15:38:28.600 INFO:tasks.cephadm:Adding mon.b on smithi134 2023-08-11T15:38:28.600 INFO:tasks.cephadm:Adding mon.c on smithi192 2023-08-11T15:38:28.600 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch apply mon '3;smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c' 2023-08-11T15:38:29.163 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:29.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:29 smithi032 ceph-mon[97103]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:29.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:29 smithi032 ceph-mon[97103]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:29.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:29 smithi032 ceph-mon[97103]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:38:29.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:29 smithi032 ceph-mon[97103]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:29.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:29 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2899386737' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-11T15:38:29.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:29 smithi032 ceph-mon[97103]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:30.360 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.358+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/3638601846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 msgr2=0x7fc8b4105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:30.360 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.358+0000 7fc8bb0ab700 1 --2- 172.21.15.192:0/3638601846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4105710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009910 tx=0x7fc8a8009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:30.360 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.358+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/3638601846 shutdown_connections 2023-08-11T15:38:30.360 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.358+0000 7fc8bb0ab700 1 --2- 172.21.15.192:0/3638601846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4105710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:30.360 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.358+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/3638601846 >> 172.21.15.192:0/3638601846 conn(0x7fc8b4074e20 msgr2=0x7fc8b4075230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:30.361 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.358+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/3638601846 shutdown_connections 2023-08-11T15:38:30.361 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.358+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/3638601846 wait complete. 2023-08-11T15:38:30.362 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.359+0000 7fc8bb0ab700 1 Processor -- start 2023-08-11T15:38:30.362 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.359+0000 7fc8bb0ab700 1 -- start start 2023-08-11T15:38:30.362 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.359+0000 7fc8bb0ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:30.362 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.359+0000 7fc8bb0ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4196bd0 con 0x7fc8b4105330 2023-08-11T15:38:30.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.359+0000 7fc8b8e47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:30.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.359+0000 7fc8b8e47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.192:56142/0 (socket says 172.21.15.192:56142) 2023-08-11T15:38:30.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.359+0000 7fc8b8e47700 1 -- 172.21.15.192:0/2454643265 learned_addr learned my addr 172.21.15.192:0/2454643265 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:38:30.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.360+0000 7fc8b8e47700 1 -- 172.21.15.192:0/2454643265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80095f0 con 0x7fc8b4105330 2023-08-11T15:38:30.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.360+0000 7fc8b8e47700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4196690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8004f40 tx=0x7fc8a8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:30.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.360+0000 7fc8b1ffb700 1 -- 172.21.15.192:0/2454643265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a801d070 con 0x7fc8b4105330 2023-08-11T15:38:30.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.360+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4196e30 con 0x7fc8b4105330 2023-08-11T15:38:30.364 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.360+0000 7fc8b1ffb700 1 -- 172.21.15.192:0/2454643265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a8004370 con 0x7fc8b4105330 2023-08-11T15:38:30.364 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.360+0000 7fc8b1ffb700 1 -- 172.21.15.192:0/2454643265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a8003970 con 0x7fc8b4105330 2023-08-11T15:38:30.364 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.360+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b41972b0 con 0x7fc8b4105330 2023-08-11T15:38:30.365 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.361+0000 7fc8b1ffb700 1 -- 172.21.15.192:0/2454643265 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fc8a8003c30 con 0x7fc8b4105330 2023-08-11T15:38:30.365 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.361+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b41974f0 con 0x7fc8b4105330 2023-08-11T15:38:30.365 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.361+0000 7fc8b1ffb700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8a40381b0 0x7fc8a403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:30.365 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.361+0000 7fc8b1ffb700 1 -- 172.21.15.192:0/2454643265 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc8a804bd80 con 0x7fc8b4105330 2023-08-11T15:38:30.365 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.362+0000 7fc8b3fff700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8a40381b0 0x7fc8a403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:30.365 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.362+0000 7fc8b3fff700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8a40381b0 0x7fc8a403a670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0006fd0 tx=0x7fc8a0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:30.368 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.367+0000 7fc8b1ffb700 1 -- 172.21.15.192:0/2454643265 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc8a8029950 con 0x7fc8b4105330 2023-08-11T15:38:30.561 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.559+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8b41095a0 con 0x7fc8a40381b0 2023-08-11T15:38:30.568 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.567+0000 7fc8b1ffb700 1 -- 172.21.15.192:0/2454643265 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc8b41095a0 con 0x7fc8a40381b0 2023-08-11T15:38:30.568 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2023-08-11T15:38:30.570 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8a40381b0 msgr2=0x7fc8a403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:30.570 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8a40381b0 0x7fc8a403a670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0006fd0 tx=0x7fc8a0006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:30.570 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 msgr2=0x7fc8b4196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:30.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4196690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8004f40 tx=0x7fc8a8005020 comp rx=0 tx=0).stop 2023-08-11T15:38:30.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 shutdown_connections 2023-08-11T15:38:30.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8a40381b0 0x7fc8a403a670 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:30.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 --2- 172.21.15.192:0/2454643265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4105330 0x7fc8b4196690 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:30.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 >> 172.21.15.192:0/2454643265 conn(0x7fc8b4074e20 msgr2=0x7fc8b40fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:30.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.568+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 shutdown_connections 2023-08-11T15:38:30.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:30.569+0000 7fc8bb0ab700 1 -- 172.21.15.192:0/2454643265 wait complete. 2023-08-11T15:38:30.938 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.b.service 2023-08-11T15:38:30.942 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.c.service 2023-08-11T15:38:30.946 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-11T15:38:30.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph mon dump -f json 2023-08-11T15:38:30.975 INFO:journalctl@ceph.mon.c.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:38:30.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:38:31.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:31.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:31.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:31.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:31.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:31.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:31.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:31.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:31.609 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:32.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:32 smithi032 ceph-mon[97103]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:32.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:32 smithi032 ceph-mon[97103]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c;count:3 2023-08-11T15:38:32.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:32 smithi032 ceph-mon[97103]: Deploying daemon mon.c on smithi192 2023-08-11T15:38:33.277 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.274+0000 7f3c5638a700 1 -- 172.21.15.192:0/3042206344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 msgr2=0x7f3c50105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:33.277 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.274+0000 7f3c5638a700 1 --2- 172.21.15.192:0/3042206344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c50105710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009910 tx=0x7f3c40009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:33.277 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.274+0000 7f3c5638a700 1 -- 172.21.15.192:0/3042206344 shutdown_connections 2023-08-11T15:38:33.277 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.274+0000 7f3c5638a700 1 --2- 172.21.15.192:0/3042206344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c50105710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:33.278 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.275+0000 7f3c5638a700 1 -- 172.21.15.192:0/3042206344 >> 172.21.15.192:0/3042206344 conn(0x7f3c50074e20 msgr2=0x7f3c50075230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:33.278 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.275+0000 7f3c5638a700 1 -- 172.21.15.192:0/3042206344 shutdown_connections 2023-08-11T15:38:33.278 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.275+0000 7f3c5638a700 1 -- 172.21.15.192:0/3042206344 wait complete. 2023-08-11T15:38:33.278 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.275+0000 7f3c5638a700 1 Processor -- start 2023-08-11T15:38:33.278 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.275+0000 7f3c5638a700 1 -- start start 2023-08-11T15:38:33.279 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.276+0000 7f3c5638a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c5019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:33.279 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.276+0000 7f3c5638a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5019bb60 con 0x7f3c50105330 2023-08-11T15:38:33.280 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.276+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c5019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:33.280 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.276+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c5019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.192:56148/0 (socket says 172.21.15.192:56148) 2023-08-11T15:38:33.280 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.276+0000 7f3c4ffff700 1 -- 172.21.15.192:0/3548091670 learned_addr learned my addr 172.21.15.192:0/3548091670 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:38:33.280 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.277+0000 7f3c4ffff700 1 -- 172.21.15.192:0/3548091670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400095f0 con 0x7f3c50105330 2023-08-11T15:38:33.280 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.277+0000 7f3c4ffff700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c5019b620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c40004f40 tx=0x7f3c40005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:33.280 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.277+0000 7f3c4d7fa700 1 -- 172.21.15.192:0/3548091670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c4001d070 con 0x7f3c50105330 2023-08-11T15:38:33.281 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.277+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5019bdc0 con 0x7f3c50105330 2023-08-11T15:38:33.281 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.277+0000 7f3c4d7fa700 1 -- 172.21.15.192:0/3548091670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c40004370 con 0x7f3c50105330 2023-08-11T15:38:33.281 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.277+0000 7f3c4d7fa700 1 -- 172.21.15.192:0/3548091670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c40003970 con 0x7f3c50105330 2023-08-11T15:38:33.281 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.277+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c501956a0 con 0x7f3c50105330 2023-08-11T15:38:33.281 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.278+0000 7f3c4d7fa700 1 -- 172.21.15.192:0/3548091670 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f3c40003be0 con 0x7f3c50105330 2023-08-11T15:38:33.282 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.278+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c50061f50 con 0x7f3c50105330 2023-08-11T15:38:33.282 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.278+0000 7f3c4d7fa700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c38038200 0x7f3c3803a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:33.282 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.278+0000 7f3c4d7fa700 1 -- 172.21.15.192:0/3548091670 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3c4004c090 con 0x7f3c50105330 2023-08-11T15:38:33.283 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.279+0000 7f3c4f7fe700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c38038200 0x7f3c3803a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:33.283 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.279+0000 7f3c4f7fe700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c38038200 0x7f3c3803a6c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3c44006fd0 tx=0x7f3c44006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:33.285 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.284+0000 7f3c4d7fa700 1 -- 172.21.15.192:0/3548091670 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3c40029950 con 0x7f3c50105330 2023-08-11T15:38:33.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:33 smithi032 ceph-mon[97103]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:33.591 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.589+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3c501095a0 con 0x7f3c50105330 2023-08-11T15:38:33.591 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.590+0000 7f3c4d7fa700 1 -- 172.21.15.192:0/3548091670 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f3c40026070 con 0x7f3c50105330 2023-08-11T15:38:33.592 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:38:33.592 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","modified":"2023-08-11T15:36:53.074390Z","created":"2023-08-11T15:36:53.074390Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-08-11T15:38:33.594 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c38038200 msgr2=0x7f3c3803a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:33.594 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c38038200 0x7f3c3803a6c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3c44006fd0 tx=0x7f3c44006e40 comp rx=0 tx=0).stop 2023-08-11T15:38:33.594 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 msgr2=0x7f3c5019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:33.594 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c5019b620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c40004f40 tx=0x7f3c40005020 comp rx=0 tx=0).stop 2023-08-11T15:38:33.595 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 shutdown_connections 2023-08-11T15:38:33.595 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c38038200 0x7f3c3803a6c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:33.595 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 --2- 172.21.15.192:0/3548091670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c50105330 0x7f3c5019b620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:33.595 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.592+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 >> 172.21.15.192:0/3548091670 conn(0x7f3c50074e20 msgr2=0x7f3c500feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:33.595 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.593+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 shutdown_connections 2023-08-11T15:38:33.595 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:33.593+0000 7f3c5638a700 1 -- 172.21.15.192:0/3548091670 wait complete. 2023-08-11T15:38:33.596 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2023-08-11T15:38:33.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:33 smithi192 systemd[1]: Starting Ceph mon.c for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:38:33.941 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:33 smithi192 podman[130745]: 2023-08-11T15:38:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: set uid:gid to 167:167 (ceph:ceph) 2023-08-11T15:38:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev), process ceph-mon, pid 7 2023-08-11T15:38:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: pidfile_write: ignore empty --pid-file 2023-08-11T15:38:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: load: jerasure load: lrc 2023-08-11T15:38:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: RocksDB version: 7.9.2 2023-08-11T15:38:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Git sha 0 2023-08-11T15:38:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Compile date 2023-08-10 15:31:53 2023-08-11T15:38:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: DB SUMMARY 2023-08-11T15:38:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: DB Session ID: XEHP7EFCERBLMXXFOM5A 2023-08-11T15:38:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: CURRENT file: CURRENT 2023-08-11T15:38:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: IDENTITY file: IDENTITY 2023-08-11T15:38:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-11T15:38:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-08-11T15:38:34.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-08-11T15:38:34.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.error_if_exists: 0 2023-08-11T15:38:34.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.create_if_missing: 0 2023-08-11T15:38:34.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.paranoid_checks: 1 2023-08-11T15:38:34.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-11T15:38:34.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-11T15:38:34.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.env: 0x555bbc5dfda0 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.fs: PosixFileSystem 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.info_log: 0x555bbe9e9460 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_file_opening_threads: 16 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.statistics: (nil) 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.use_fsync: 0 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_log_file_size: 0 2023-08-11T15:38:34.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-11T15:38:34.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-11T15:38:34.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.keep_log_file_num: 1000 2023-08-11T15:38:34.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.recycle_log_file_num: 0 2023-08-11T15:38:34.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.allow_fallocate: 1 2023-08-11T15:38:34.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.allow_mmap_reads: 0 2023-08-11T15:38:34.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.allow_mmap_writes: 0 2023-08-11T15:38:34.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.use_direct_reads: 0 2023-08-11T15:38:34.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-11T15:38:34.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.create_missing_column_families: 0 2023-08-11T15:38:34.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.db_log_dir: 2023-08-11T15:38:34.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.wal_dir: 2023-08-11T15:38:34.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-11T15:38:34.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-11T15:38:34.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-11T15:38:34.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-11T15:38:34.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-11T15:38:34.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-11T15:38:34.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.advise_random_on_open: 1 2023-08-11T15:38:34.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.db_write_buffer_size: 0 2023-08-11T15:38:34.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.write_buffer_manager: 0x555bbe9a23c0 2023-08-11T15:38:34.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-11T15:38:34.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-11T15:38:34.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-11T15:38:34.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.rate_limiter: (nil) 2023-08-11T15:38:34.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-11T15:38:34.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.wal_recovery_mode: 2 2023-08-11T15:38:34.592 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.enable_thread_tracking: 0 2023-08-11T15:38:34.592 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.enable_pipelined_write: 0 2023-08-11T15:38:34.592 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.unordered_write: 0 2023-08-11T15:38:34.592 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-11T15:38:34.592 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-11T15:38:34.592 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-11T15:38:34.592 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.row_cache: None 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.wal_filter: None 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.allow_ingest_behind: 0 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.two_write_queues: 0 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.manual_wal_flush: 0 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.wal_compression: 0 2023-08-11T15:38:34.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.atomic_flush: 0 2023-08-11T15:38:34.594 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-11T15:38:34.594 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-11T15:38:34.594 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-11T15:38:34.594 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.log_readahead_size: 0 2023-08-11T15:38:34.594 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-11T15:38:34.594 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.best_efforts_recovery: 0 2023-08-11T15:38:34.594 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-11T15:38:34.595 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-11T15:38:34.595 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.allow_data_in_errors: 0 2023-08-11T15:38:34.595 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.db_host_id: __hostname__ 2023-08-11T15:38:34.596 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-11T15:38:34.596 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_background_jobs: 2 2023-08-11T15:38:34.596 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_background_compactions: -1 2023-08-11T15:38:34.597 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_subcompactions: 1 2023-08-11T15:38:34.597 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-11T15:38:34.597 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-11T15:38:34.597 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-11T15:38:34.597 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_total_wal_size: 0 2023-08-11T15:38:34.597 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-11T15:38:34.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-11T15:38:34.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-11T15:38:34.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-11T15:38:34.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_open_files: -1 2023-08-11T15:38:34.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bytes_per_sync: 0 2023-08-11T15:38:34.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-11T15:38:34.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-11T15:38:34.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_readahead_size: 0 2023-08-11T15:38:34.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_background_flushes: -1 2023-08-11T15:38:34.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Compression algorithms supported: 2023-08-11T15:38:34.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-11T15:38:34.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kZSTD supported: 0 2023-08-11T15:38:34.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kXpressCompression supported: 0 2023-08-11T15:38:34.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kLZ4HCCompression supported: 1 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kZlibCompression supported: 1 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kSnappyCompression supported: 1 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kLZ4Compression supported: 1 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: kBZip2Compression supported: 0 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-08-11T15:38:34.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-11T15:38:34.601 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-11T15:38:34.601 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.merge_operator: 2023-08-11T15:38:34.601 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_filter: None 2023-08-11T15:38:34.601 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_filter_factory: None 2023-08-11T15:38:34.601 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.sst_partitioner_factory: None 2023-08-11T15:38:34.601 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-11T15:38:34.601 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555bbe9e9560) 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks: 1 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_top_level_index_and_filter: 1 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_type: 0 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_index_type: 0 2023-08-11T15:38:34.602 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_shortening: 1 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: checksum: 4 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: no_block_cache: 0 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache: 0x555bbea411f0 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_name: BinnedLRUCache 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_options: 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: capacity : 536870912 2023-08-11T15:38:34.603 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_shard_bits : 4 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: strict_capacity_limit : 0 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: high_pri_pool_ratio: 0.000 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_compressed: (nil) 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: persistent_cache: (nil) 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size: 4096 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size_deviation: 10 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_restart_interval: 16 2023-08-11T15:38:34.604 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_block_restart_interval: 1 2023-08-11T15:38:34.605 INFO:journalctl@ceph.mon.c.smithi192.stdout: metadata_block_size: 4096 2023-08-11T15:38:34.605 INFO:journalctl@ceph.mon.c.smithi192.stdout: partition_filters: 0 2023-08-11T15:38:34.605 INFO:journalctl@ceph.mon.c.smithi192.stdout: use_delta_encoding: 1 2023-08-11T15:38:34.605 INFO:journalctl@ceph.mon.c.smithi192.stdout: filter_policy: bloomfilter 2023-08-11T15:38:34.605 INFO:journalctl@ceph.mon.c.smithi192.stdout: whole_key_filtering: 1 2023-08-11T15:38:34.605 INFO:journalctl@ceph.mon.c.smithi192.stdout: verify_compression: 0 2023-08-11T15:38:34.605 INFO:journalctl@ceph.mon.c.smithi192.stdout: read_amp_bytes_per_bit: 0 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout: format_version: 5 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_index_compression: 1 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_align: 0 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout: max_auto_readahead_size: 262144 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout: prepopulate_block_cache: 0 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout: initial_auto_readahead_size: 8192 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2023-08-11T15:38:34.606 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.write_buffer_size: 33554432 2023-08-11T15:38:34.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_write_buffer_number: 2 2023-08-11T15:38:34.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression: NoCompression 2023-08-11T15:38:34.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression: Disabled 2023-08-11T15:38:34.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.prefix_extractor: nullptr 2023-08-11T15:38:34.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-11T15:38:34.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.num_levels: 7 2023-08-11T15:38:34.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-11T15:38:34.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-11T15:38:34.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-11T15:38:34.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-11T15:38:34.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-11T15:38:34.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-11T15:38:34.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-11T15:38:34.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.level: 32767 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.strategy: 0 2023-08-11T15:38:34.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-11T15:38:34.610 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-11T15:38:34.610 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-11T15:38:34.610 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-11T15:38:34.610 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.enabled: false 2023-08-11T15:38:34.610 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-11T15:38:34.610 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-11T15:38:34.610 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.target_file_size_base: 67108864 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-11T15:38:34.611 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-11T15:38:34.612 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-11T15:38:34.612 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-11T15:38:34.612 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-11T15:38:34.612 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-11T15:38:34.612 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-11T15:38:34.612 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-11T15:38:34.612 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-11T15:38:34.613 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-11T15:38:34.613 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.arena_block_size: 1048576 2023-08-11T15:38:34.613 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-11T15:38:34.613 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-11T15:38:34.613 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.disable_auto_compactions: 0 2023-08-11T15:38:34.613 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-11T15:38:34.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-11T15:38:34.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-11T15:38:34.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-11T15:38:34.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-11T15:38:34.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-11T15:38:34.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-11T15:38:34.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.table_properties_collectors: 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.inplace_update_support: 0 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-11T15:38:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.bloom_locality: 0 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.max_successive_merges: 0 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.paranoid_file_checks: 0 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.force_consistency_checks: 1 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.report_bg_io_stats: 0 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.ttl: 2592000 2023-08-11T15:38:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-11T15:38:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-11T15:38:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-11T15:38:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.enable_blob_files: false 2023-08-11T15:38:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.min_blob_size: 0 2023-08-11T15:38:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.blob_file_size: 268435456 2023-08-11T15:38:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-11T15:38:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.blob_file_starting_level: 0 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-11T15:38:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 348f8faf-aeca-4624-b895-86648a56c896 2023-08-11T15:38:34.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768314125093, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-11T15:38:34.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-11T15:38:34.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768314125507, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1691768314, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "348f8faf-aeca-4624-b895-86648a56c896", "db_session_id": "XEHP7EFCERBLMXXFOM5A", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-11T15:38:34.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768314125574, "job": 1, "event": "recovery_finished"} 2023-08-11T15:38:34.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-11T15:38:34.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-11T15:38:34.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555bbeade000 2023-08-11T15:38:34.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: DB pointer 0x555bbeaca000 2023-08-11T15:38:34.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-11T15:38:34.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-11T15:38:34.620 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** DB Stats ** 2023-08-11T15:38:34.620 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T15:38:34.620 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-11T15:38:34.620 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T15:38:34.621 INFO:journalctl@ceph.mon.c.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-11T15:38:34.622 INFO:journalctl@ceph.mon.c.smithi192.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T15:38:34.622 INFO:journalctl@ceph.mon.c.smithi192.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T15:38:34.622 INFO:journalctl@ceph.mon.c.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T15:38:34.622 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-08-11T15:38:34.622 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2023-08-11T15:38:34.622 INFO:journalctl@ceph.mon.c.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T15:38:34.622 INFO:journalctl@ceph.mon.c.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-11T15:38:34.623 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-11T15:38:34.624 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-11T15:38:34.624 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-11T15:38:34.624 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T15:38:34.624 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T15:38:34.624 INFO:journalctl@ceph.mon.c.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-11T15:38:34.624 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache BinnedLRUCache@0x555bbea411f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-08-11T15:38:34.624 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-11T15:38:34.625 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-08-11T15:38:34.625 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-11T15:38:34.625 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-08-11T15:38:34.625 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: using public_addr v2:172.21.15.192:0/0 -> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2023-08-11T15:38:34.625 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: starting mon.c rank -1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:34.625 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(???) e0 preinit fsid cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:34.625 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).mds e1 new map 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).mds e1 print_map 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout: e1 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout: legacy client fscid: -1 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout: No filesystems configured 2023-08-11T15:38:34.626 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-11T15:38:34.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-11T15:38:34.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-11T15:38:34.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-11T15:38:34.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-11T15:38:34.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-11T15:38:34.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-11T15:38:34.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: Added host smithi192 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:34.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:38:34.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:34.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2899386737' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:38:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2899386737' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-11T15:38:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:34.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:34.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:34.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c;count:3 2023-08-11T15:38:34.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: Deploying daemon mon.c on smithi192 2023-08-11T15:38:34.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:34.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-11T15:38:34.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-08-11T15:38:34.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expand map: {default=false} 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta from 'false' to 'false' 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expanded map: {default=false} 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expand map: {default=info} 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta from 'info' to 'info' 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expanded map: {default=info} 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expand map: {default=daemon} 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-11T15:38:34.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expanded map: {default=daemon} 2023-08-11T15:38:34.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expand map: {default=debug} 2023-08-11T15:38:34.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta from 'debug' to 'debug' 2023-08-11T15:38:34.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 ceph-mon[130806]: expand_channel_meta expanded map: {default=debug} 2023-08-11T15:38:34.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 bash[130745]: f9c31a36d07c69f2877bd87c83c3065d8d0c9eba207e2e48dad7e5f4abeb5658 2023-08-11T15:38:34.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:34 smithi192 systemd[1]: Started Ceph mon.c for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:38:35.216 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-11T15:38:35.217 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph mon dump -f json 2023-08-11T15:38:35.780 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:38:39.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:39.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:38:39.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: mon.a calling monitor election 2023-08-11T15:38:39.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:39.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: mon.c calling monitor election 2023-08-11T15:38:39.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:39.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T15:38:39.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: monmap e2: 2 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:39.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: fsmap 2023-08-11T15:38:39.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:39.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: mgrmap e13: x(active, since 45s) 2023-08-11T15:38:39.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: overall HEALTH_OK 2023-08-11T15:38:39.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:39.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:39.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:39.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:39.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:38:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: mon.a calling monitor election 2023-08-11T15:38:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: mon.c calling monitor election 2023-08-11T15:38:39.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:39.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:39.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T15:38:39.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: monmap e2: 2 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:39.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: fsmap 2023-08-11T15:38:39.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:39.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: mgrmap e13: x(active, since 45s) 2023-08-11T15:38:39.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: overall HEALTH_OK 2023-08-11T15:38:39.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:39.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:39.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:39.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:39.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:39.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.920+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1794628415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 msgr2=0x7f82d4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:39.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.920+0000 7f82f0ddd700 1 --2- 172.21.15.192:0/1794628415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 0x7f82d4005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f82ec1003f0 tx=0x7f82dc00feb0 comp rx=0 tx=0).stop 2023-08-11T15:38:39.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.920+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1794628415 shutdown_connections 2023-08-11T15:38:39.923 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.920+0000 7f82f0ddd700 1 --2- 172.21.15.192:0/1794628415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 0x7f82d4005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:39.923 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.920+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1794628415 >> 172.21.15.192:0/1794628415 conn(0x7f82ec0fbf90 msgr2=0x7f82ec0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:39.923 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.920+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1794628415 shutdown_connections 2023-08-11T15:38:39.923 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.920+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1794628415 wait complete. 2023-08-11T15:38:39.924 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.922+0000 7f82f0ddd700 1 Processor -- start 2023-08-11T15:38:39.924 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.922+0000 7f82f0ddd700 1 -- start start 2023-08-11T15:38:39.924 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82f0ddd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 0x7f82ec072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:39.924 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82f0ddd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 0x7f82ec0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:39.925 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82f0ddd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec074ba0 con 0x7f82d40035c0 2023-08-11T15:38:39.925 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82f0ddd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec1a1ad0 con 0x7f82ec073160 2023-08-11T15:38:39.926 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82e9d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 0x7f82ec0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:39.926 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82e9d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 0x7f82ec0735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57372/0 (socket says 172.21.15.192:57372) 2023-08-11T15:38:39.928 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82e9d9b700 1 -- 172.21.15.192:0/1102212262 learned_addr learned my addr 172.21.15.192:0/1102212262 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:38:39.928 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82e9d9b700 1 -- 172.21.15.192:0/1102212262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 msgr2=0x7f82ec0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-08-11T15:38:39.928 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82e9d9b700 1 -- 172.21.15.192:0/1102212262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 msgr2=0x7f82ec0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-11T15:38:39.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82e9d9b700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 0x7f82ec0735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T15:38:39.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82e9d9b700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 0x7f82ec0735e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:38:39.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.923+0000 7f82ea59c700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 0x7f82ec072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:39.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82ea59c700 1 -- 172.21.15.192:0/1102212262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 msgr2=0x7f82ec0735e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:38:39.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82ea59c700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 0x7f82ec0735e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:39.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82ea59c700 1 -- 172.21.15.192:0/1102212262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc00f900 con 0x7f82d40035c0 2023-08-11T15:38:39.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82ea59c700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 0x7f82ec072c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f82dc010010 tx=0x7f82dc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:39.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82db7fe700 1 -- 172.21.15.192:0/1102212262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f82dc013960 con 0x7f82d40035c0 2023-08-11T15:38:39.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82db7fe700 1 -- 172.21.15.192:0/1102212262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82dc004030 con 0x7f82d40035c0 2023-08-11T15:38:39.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82db7fe700 1 -- 172.21.15.192:0/1102212262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f82dc003a80 con 0x7f82d40035c0 2023-08-11T15:38:39.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec1a1cd0 con 0x7f82d40035c0 2023-08-11T15:38:39.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.924+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ec1a21a0 con 0x7f82d40035c0 2023-08-11T15:38:39.931 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.925+0000 7f82db7fe700 1 -- 172.21.15.192:0/1102212262 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f82dc003cf0 con 0x7f82d40035c0 2023-08-11T15:38:39.931 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.925+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ec061f50 con 0x7f82d40035c0 2023-08-11T15:38:39.931 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.925+0000 7f82db7fe700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f82cc037ff0 0x7f82cc03a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:39.931 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.926+0000 7f82db7fe700 1 -- 172.21.15.192:0/1102212262 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f82dc000e40 con 0x7f82d40035c0 2023-08-11T15:38:39.932 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.926+0000 7f82e9d9b700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f82cc037ff0 0x7f82cc03a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:39.932 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.926+0000 7f82e9d9b700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f82cc037ff0 0x7f82cc03a4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009180 tx=0x7f82e0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:39.931+0000 7f82db7fe700 1 -- 172.21.15.192:0/1102212262 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f82dc01a750 con 0x7f82d40035c0 2023-08-11T15:38:40.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.198+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f82ec074150 con 0x7f82d40035c0 2023-08-11T15:38:40.200 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.199+0000 7f82db7fe700 1 -- 172.21.15.192:0/1102212262 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f82dc01a390 con 0x7f82d40035c0 2023-08-11T15:38:40.200 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:38:40.201 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","modified":"2023-08-11T15:38:34.156258Z","created":"2023-08-11T15:36:53.074390Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-08-11T15:38:40.202 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f82cc037ff0 msgr2=0x7f82cc03a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:40.203 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f82cc037ff0 0x7f82cc03a4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009180 tx=0x7f82e0008040 comp rx=0 tx=0).stop 2023-08-11T15:38:40.203 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 msgr2=0x7f82ec072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:40.203 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 0x7f82ec072c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f82dc010010 tx=0x7f82dc0047a0 comp rx=0 tx=0).stop 2023-08-11T15:38:40.203 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 shutdown_connections 2023-08-11T15:38:40.203 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f82cc037ff0 0x7f82cc03a4b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:40.204 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82d40035c0 0x7f82ec072c20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:40.204 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 --2- 172.21.15.192:0/1102212262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ec073160 0x7f82ec0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:40.204 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 >> 172.21.15.192:0/1102212262 conn(0x7f82ec0fbf90 msgr2=0x7f82ec104920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:40.204 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 shutdown_connections 2023-08-11T15:38:40.204 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:40.201+0000 7f82f0ddd700 1 -- 172.21.15.192:0/1102212262 wait complete. 2023-08-11T15:38:40.205 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2023-08-11T15:38:40.523 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:40 smithi192 ceph-mon[130806]: Deploying daemon mon.b on smithi134 2023-08-11T15:38:40.523 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:40 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:40.524 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:40 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/1102212262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T15:38:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:40 smithi032 ceph-mon[97103]: Deploying daemon mon.b on smithi134 2023-08-11T15:38:40.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:40 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:40.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:40 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/1102212262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T15:38:41.543 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-11T15:38:41.774 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph mon dump -f json 2023-08-11T15:38:41.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:41 smithi192 ceph-mon[130806]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:41.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:41 smithi032 ceph-mon[97103]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:41.776 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:38:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:38:41.156+0000 7f6120ac6700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-11T15:38:42.165 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:41 smithi134 systemd[1]: Starting Ceph mon.b for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:38:42.339 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:38:42.453 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 podman[130408]: 2023-08-11T15:38:42.453 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 bash[130408]: 266dc0a763d5482d3705dfc98ca723a10ff48d284a3b9dc636d8c90e27de9483 2023-08-11T15:38:42.453 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: set uid:gid to 167:167 (ceph:ceph) 2023-08-11T15:38:42.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev), process ceph-mon, pid 7 2023-08-11T15:38:42.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pidfile_write: ignore empty --pid-file 2023-08-11T15:38:42.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: load: jerasure load: lrc 2023-08-11T15:38:42.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: RocksDB version: 7.9.2 2023-08-11T15:38:42.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Git sha 0 2023-08-11T15:38:42.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Compile date 2023-08-10 15:31:53 2023-08-11T15:38:42.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: DB SUMMARY 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: DB Session ID: ADFDMJNIF4PIKQ4APDR2 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: CURRENT file: CURRENT 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: IDENTITY file: IDENTITY 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.error_if_exists: 0 2023-08-11T15:38:42.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.create_if_missing: 0 2023-08-11T15:38:42.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.paranoid_checks: 1 2023-08-11T15:38:42.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-11T15:38:42.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-11T15:38:42.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-11T15:38:42.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.env: 0x5629643d7da0 2023-08-11T15:38:42.457 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.fs: PosixFileSystem 2023-08-11T15:38:42.457 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.info_log: 0x562965ecf460 2023-08-11T15:38:42.457 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_file_opening_threads: 16 2023-08-11T15:38:42.457 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.statistics: (nil) 2023-08-11T15:38:42.457 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.use_fsync: 0 2023-08-11T15:38:42.457 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_log_file_size: 0 2023-08-11T15:38:42.457 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.keep_log_file_num: 1000 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.recycle_log_file_num: 0 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.allow_fallocate: 1 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.allow_mmap_reads: 0 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.allow_mmap_writes: 0 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.use_direct_reads: 0 2023-08-11T15:38:42.458 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-11T15:38:42.459 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.create_missing_column_families: 0 2023-08-11T15:38:42.459 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.db_log_dir: 2023-08-11T15:38:42.459 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.wal_dir: 2023-08-11T15:38:42.459 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-11T15:38:42.459 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-11T15:38:42.459 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-11T15:38:42.459 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.advise_random_on_open: 1 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.db_write_buffer_size: 0 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.write_buffer_manager: 0x562965e883c0 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-11T15:38:42.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-11T15:38:42.461 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.rate_limiter: (nil) 2023-08-11T15:38:42.461 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-11T15:38:42.461 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.wal_recovery_mode: 2 2023-08-11T15:38:42.461 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.enable_thread_tracking: 0 2023-08-11T15:38:42.461 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.enable_pipelined_write: 0 2023-08-11T15:38:42.461 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.unordered_write: 0 2023-08-11T15:38:42.461 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.row_cache: None 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.wal_filter: None 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.allow_ingest_behind: 0 2023-08-11T15:38:42.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.two_write_queues: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.manual_wal_flush: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.wal_compression: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.atomic_flush: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.log_readahead_size: 0 2023-08-11T15:38:42.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-11T15:38:42.464 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.best_efforts_recovery: 0 2023-08-11T15:38:42.464 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-11T15:38:42.464 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-11T15:38:42.465 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.allow_data_in_errors: 0 2023-08-11T15:38:42.465 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.db_host_id: __hostname__ 2023-08-11T15:38:42.465 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-11T15:38:42.465 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_background_jobs: 2 2023-08-11T15:38:42.466 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_background_compactions: -1 2023-08-11T15:38:42.466 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_subcompactions: 1 2023-08-11T15:38:42.466 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-11T15:38:42.466 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-11T15:38:42.466 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-11T15:38:42.466 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_total_wal_size: 0 2023-08-11T15:38:42.466 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_open_files: -1 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bytes_per_sync: 0 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-11T15:38:42.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_readahead_size: 0 2023-08-11T15:38:42.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_background_flushes: -1 2023-08-11T15:38:42.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Compression algorithms supported: 2023-08-11T15:38:42.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-11T15:38:42.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kZSTD supported: 0 2023-08-11T15:38:42.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kXpressCompression supported: 0 2023-08-11T15:38:42.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kLZ4HCCompression supported: 1 2023-08-11T15:38:42.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kZlibCompression supported: 1 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kSnappyCompression supported: 1 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kLZ4Compression supported: 1 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: kBZip2Compression supported: 0 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-11T15:38:42.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-11T15:38:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.merge_operator: 2023-08-11T15:38:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_filter: None 2023-08-11T15:38:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_filter_factory: None 2023-08-11T15:38:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.sst_partitioner_factory: None 2023-08-11T15:38:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-11T15:38:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-11T15:38:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562965ecf560) 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks: 1 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_top_level_index_and_filter: 1 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_type: 0 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_index_type: 0 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_shortening: 1 2023-08-11T15:38:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: checksum: 4 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: no_block_cache: 0 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache: 0x562965f271f0 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_name: BinnedLRUCache 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_options: 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: capacity : 536870912 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_shard_bits : 4 2023-08-11T15:38:42.472 INFO:journalctl@ceph.mon.b.smithi134.stdout: strict_capacity_limit : 0 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: high_pri_pool_ratio: 0.000 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_compressed: (nil) 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: persistent_cache: (nil) 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size: 4096 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size_deviation: 10 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_restart_interval: 16 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_block_restart_interval: 1 2023-08-11T15:38:42.473 INFO:journalctl@ceph.mon.b.smithi134.stdout: metadata_block_size: 4096 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: partition_filters: 0 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: use_delta_encoding: 1 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: filter_policy: bloomfilter 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: whole_key_filtering: 1 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: verify_compression: 0 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: read_amp_bytes_per_bit: 0 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: format_version: 5 2023-08-11T15:38:42.474 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_index_compression: 1 2023-08-11T15:38:42.475 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_align: 0 2023-08-11T15:38:42.475 INFO:journalctl@ceph.mon.b.smithi134.stdout: max_auto_readahead_size: 262144 2023-08-11T15:38:42.475 INFO:journalctl@ceph.mon.b.smithi134.stdout: prepopulate_block_cache: 0 2023-08-11T15:38:42.475 INFO:journalctl@ceph.mon.b.smithi134.stdout: initial_auto_readahead_size: 8192 2023-08-11T15:38:42.475 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_file_reads_for_auto_readahead: 2 2023-08-11T15:38:42.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.write_buffer_size: 33554432 2023-08-11T15:38:42.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_write_buffer_number: 2 2023-08-11T15:38:42.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression: NoCompression 2023-08-11T15:38:42.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression: Disabled 2023-08-11T15:38:42.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.prefix_extractor: nullptr 2023-08-11T15:38:42.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-11T15:38:42.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.num_levels: 7 2023-08-11T15:38:42.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-11T15:38:42.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-11T15:38:42.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-11T15:38:42.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-11T15:38:42.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-11T15:38:42.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-11T15:38:42.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.level: 32767 2023-08-11T15:38:42.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.strategy: 0 2023-08-11T15:38:42.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-11T15:38:42.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-11T15:38:42.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-11T15:38:42.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-11T15:38:42.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.enabled: false 2023-08-11T15:38:42.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-11T15:38:42.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-11T15:38:42.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-11T15:38:42.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-11T15:38:42.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.target_file_size_base: 67108864 2023-08-11T15:38:42.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-11T15:38:42.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-11T15:38:42.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-11T15:38:42.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-11T15:38:42.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-11T15:38:42.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-11T15:38:42.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-11T15:38:42.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-11T15:38:42.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-11T15:38:42.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-11T15:38:42.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-11T15:38:42.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-11T15:38:42.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.arena_block_size: 1048576 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.disable_auto_compactions: 0 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-11T15:38:42.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-11T15:38:42.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-11T15:38:42.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-11T15:38:42.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-11T15:38:42.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-11T15:38:42.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-11T15:38:42.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-11T15:38:42.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.table_properties_collectors: 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.inplace_update_support: 0 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.bloom_locality: 0 2023-08-11T15:38:42.485 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.max_successive_merges: 0 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.paranoid_file_checks: 0 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.force_consistency_checks: 1 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.report_bg_io_stats: 0 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.ttl: 2592000 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-11T15:38:42.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-11T15:38:42.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.enable_blob_files: false 2023-08-11T15:38:42.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.min_blob_size: 0 2023-08-11T15:38:42.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.blob_file_size: 268435456 2023-08-11T15:38:42.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-11T15:38:42.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-11T15:38:42.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-11T15:38:42.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.blob_file_starting_level: 0 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 45b7b13c-5c6d-49e2-802d-9dfb348721a5 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768322372031, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-11T15:38:42.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-11T15:38:42.489 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768322372577, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1691768322, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "45b7b13c-5c6d-49e2-802d-9dfb348721a5", "db_session_id": "ADFDMJNIF4PIKQ4APDR2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-11T15:38:42.489 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691768322372666, "job": 1, "event": "recovery_finished"} 2023-08-11T15:38:42.489 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-11T15:38:42.489 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-11T15:38:42.489 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562965fc4000 2023-08-11T15:38:42.489 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: DB pointer 0x562965fb0000 2023-08-11T15:38:42.489 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** DB Stats ** 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-11T15:38:42.490 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:38:42.491 INFO:journalctl@ceph.mon.b.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:38:42.492 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-08-11T15:38:42.492 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-08-11T15:38:42.492 INFO:journalctl@ceph.mon.b.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T15:38:42.492 INFO:journalctl@ceph.mon.b.smithi134.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-11T15:38:42.492 INFO:journalctl@ceph.mon.b.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T15:38:42.493 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-08-11T15:38:42.493 INFO:journalctl@ceph.mon.b.smithi134.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-11T15:38:42.493 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-08-11T15:38:42.493 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T15:38:42.493 INFO:journalctl@ceph.mon.b.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-11T15:38:42.494 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-11T15:38:42.494 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-11T15:38:42.494 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-11T15:38:42.494 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-11T15:38:42.494 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T15:38:42.494 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T15:38:42.494 INFO:journalctl@ceph.mon.b.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache BinnedLRUCache@0x562965f271f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: starting mon.b rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:42.495 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(???) e0 preinit fsid cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:42.496 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).mds e1 new map 2023-08-11T15:38:42.496 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).mds e1 print_map 2023-08-11T15:38:42.496 INFO:journalctl@ceph.mon.b.smithi134.stdout: e1 2023-08-11T15:38:42.496 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T15:38:42.496 INFO:journalctl@ceph.mon.b.smithi134.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T15:38:42.496 INFO:journalctl@ceph.mon.b.smithi134.stdout: legacy client fscid: -1 2023-08-11T15:38:42.496 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout: No filesystems configured 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-11T15:38:42.497 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-11T15:38:42.498 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:38:42.498 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:38:42.498 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T15:38:42.498 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.498 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.498 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Added host smithi192 2023-08-11T15:38:42.498 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.499 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2899386737' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-11T15:38:42.500 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.501 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.501 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.501 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:42.501 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:42.501 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:38:42.501 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.501 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2899386737' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.502 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c;count:3 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Deploying daemon mon.c on smithi192 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:38:42.503 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:42.504 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.a calling monitor election 2023-08-11T15:38:42.504 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:42.504 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.504 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:42.504 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.c calling monitor election 2023-08-11T15:38:42.504 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:42.504 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: monmap e2: 2 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: fsmap 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mgrmap e13: x(active, since 45s) 2023-08-11T15:38:42.505 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: overall HEALTH_OK 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: Deploying daemon mon.b on smithi134 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:42.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/1102212262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expand map: {default=false} 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta from 'false' to 'false' 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expanded map: {default=false} 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expand map: {default=info} 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta from 'info' to 'info' 2023-08-11T15:38:42.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expanded map: {default=info} 2023-08-11T15:38:42.508 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expand map: {default=daemon} 2023-08-11T15:38:42.508 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-11T15:38:42.508 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expanded map: {default=daemon} 2023-08-11T15:38:42.508 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expand map: {default=debug} 2023-08-11T15:38:42.508 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta from 'debug' to 'debug' 2023-08-11T15:38:42.508 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: expand_channel_meta expanded map: {default=debug} 2023-08-11T15:38:42.508 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 ceph-mon[130447]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-08-11T15:38:42.509 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:42 smithi134 systemd[1]: Started Ceph mon.b for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:38:47.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:38:47.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: mon.c calling monitor election 2023-08-11T15:38:47.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: mon.a calling monitor election 2023-08-11T15:38:47.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:47.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:47.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:47.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T15:38:47.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:47.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: fsmap 2023-08-11T15:38:47.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:47.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: mgrmap e13: x(active, since 53s) 2023-08-11T15:38:47.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-11T15:38:47.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-11T15:38:47.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-11T15:38:47.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-08-11T15:38:47.810 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:47.810 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:47.810 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:47.829 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.827+0000 7fe2689d5700 1 -- 172.21.15.192:0/3339666189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 msgr2=0x7fe24c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:47.829 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.827+0000 7fe2689d5700 1 --2- 172.21.15.192:0/3339666189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 0x7fe24c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2640ff300 tx=0x7fe25800fe60 comp rx=0 tx=0).stop 2023-08-11T15:38:47.829 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.827+0000 7fe2689d5700 1 -- 172.21.15.192:0/3339666189 shutdown_connections 2023-08-11T15:38:47.829 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.827+0000 7fe2689d5700 1 --2- 172.21.15.192:0/3339666189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 0x7fe24c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:47.829 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.827+0000 7fe2689d5700 1 -- 172.21.15.192:0/3339666189 >> 172.21.15.192:0/3339666189 conn(0x7fe2640faee0 msgr2=0x7fe2640fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:47.830 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.827+0000 7fe2689d5700 1 -- 172.21.15.192:0/3339666189 shutdown_connections 2023-08-11T15:38:47.830 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.827+0000 7fe2689d5700 1 -- 172.21.15.192:0/3339666189 wait complete. 2023-08-11T15:38:47.835 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.828+0000 7fe2689d5700 1 Processor -- start 2023-08-11T15:38:47.840 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.828+0000 7fe2689d5700 1 -- start start 2023-08-11T15:38:47.840 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe2689d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 0x7fe264196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:47.840 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe2689d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 0x7fe264197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:47.840 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe2689d5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 0x7fe2641a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:47.841 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe2689d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2641a24f0 con 0x7fe24c0035c0 2023-08-11T15:38:47.841 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe2689d5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2641a2660 con 0x7fe26419f9b0 2023-08-11T15:38:47.841 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe2689d5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2641a2820 con 0x7fe264196e80 2023-08-11T15:38:47.841 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe261d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 0x7fe264197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:47.845 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe261d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 0x7fe264197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57398/0 (socket says 172.21.15.192:57398) 2023-08-11T15:38:47.845 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe261d9b700 1 -- 172.21.15.192:0/2544186641 learned_addr learned my addr 172.21.15.192:0/2544186641 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:38:47.845 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe261d9b700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 msgr2=0x7fe264197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-08-11T15:38:47.845 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe261d9b700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 msgr2=0x7fe264197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-11T15:38:47.846 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe261d9b700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 0x7fe264197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T15:38:47.846 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe26259c700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 0x7fe264196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:47.846 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe261d9b700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 0x7fe264197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:38:47.846 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe262d9d700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 0x7fe2641a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:47.846 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe262d9d700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 msgr2=0x7fe2641a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-08-11T15:38:47.847 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe262d9d700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 msgr2=0x7fe2641a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-11T15:38:47.847 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe262d9d700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 0x7fe2641a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T15:38:47.847 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe262d9d700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 0x7fe2641a1da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:38:47.847 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe26259c700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 msgr2=0x7fe264197300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:38:47.847 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe26259c700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 0x7fe264197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:47.847 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe26259c700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 msgr2=0x7fe2641a1da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:38:47.847 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe26259c700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 0x7fe2641a1da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:47.848 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.829+0000 7fe26259c700 1 -- 172.21.15.192:0/2544186641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25800f8b0 con 0x7fe24c0035c0 2023-08-11T15:38:47.848 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.830+0000 7fe26259c700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 0x7fe264196940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe258020040 tx=0x7fe258012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:47.848 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.830+0000 7fe2537fe700 1 -- 172.21.15.192:0/2544186641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258004130 con 0x7fe24c0035c0 2023-08-11T15:38:47.848 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.830+0000 7fe2537fe700 1 -- 172.21.15.192:0/2544186641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe258004290 con 0x7fe24c0035c0 2023-08-11T15:38:47.849 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.830+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2641a2aa0 con 0x7fe24c0035c0 2023-08-11T15:38:47.849 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.830+0000 7fe2537fe700 1 -- 172.21.15.192:0/2544186641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2580278b0 con 0x7fe24c0035c0 2023-08-11T15:38:47.849 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.830+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2641a2ff0 con 0x7fe24c0035c0 2023-08-11T15:38:47.850 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.831+0000 7fe2537fe700 1 -- 172.21.15.192:0/2544186641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7fe258024050 con 0x7fe24c0035c0 2023-08-11T15:38:47.850 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.831+0000 7fe2537fe700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe244038630 0x7fe24403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:47.850 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.831+0000 7fe2537fe700 1 -- 172.21.15.192:0/2544186641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe258000e40 con 0x7fe24c0035c0 2023-08-11T15:38:47.851 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.831+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe264061f50 con 0x7fe24c0035c0 2023-08-11T15:38:47.851 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.831+0000 7fe261d9b700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe244038630 0x7fe24403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:47.852 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:38:47.852 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: mon.c calling monitor election 2023-08-11T15:38:47.853 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: mon.a calling monitor election 2023-08-11T15:38:47.853 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.853 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:47.853 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.854 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:47.854 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.854 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.855 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:47.855 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.855 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:47.855 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T15:38:47.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:47.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: fsmap 2023-08-11T15:38:47.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:47.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: mgrmap e13: x(active, since 53s) 2023-08-11T15:38:47.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-11T15:38:47.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-11T15:38:47.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-11T15:38:47.858 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-08-11T15:38:47.858 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:47.858 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:47.859 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:47.859 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.832+0000 7fe261d9b700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe244038630 0x7fe24403aaf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe2640699f0 tx=0x7fe254008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:47.860 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:47.837+0000 7fe2537fe700 1 -- 172.21.15.192:0/2544186641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe258022020 con 0x7fe24c0035c0 2023-08-11T15:38:48.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.142+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe2641070e0 con 0x7fe24c0035c0 2023-08-11T15:38:48.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.142+0000 7fe2537fe700 1 -- 172.21.15.192:0/2544186641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7fe25803f090 con 0x7fe24c0035c0 2023-08-11T15:38:48.144 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-08-11T15:38:48.145 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","modified":"2023-08-11T15:38:42.401522Z","created":"2023-08-11T15:36:53.074390Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-08-11T15:38:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe244038630 msgr2=0x7fe24403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe244038630 0x7fe24403aaf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe2640699f0 tx=0x7fe254008040 comp rx=0 tx=0).stop 2023-08-11T15:38:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 msgr2=0x7fe264196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 0x7fe264196940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe258020040 tx=0x7fe258012d70 comp rx=0 tx=0).stop 2023-08-11T15:38:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 shutdown_connections 2023-08-11T15:38:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe244038630 0x7fe24403aaf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24c0035c0 0x7fe264196940 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26419f9b0 0x7fe2641a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.144+0000 7fe2689d5700 1 --2- 172.21.15.192:0/2544186641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe264196e80 0x7fe264197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.145+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 >> 172.21.15.192:0/2544186641 conn(0x7fe2640faee0 msgr2=0x7fe264100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.145+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 shutdown_connections 2023-08-11T15:38:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:48.145+0000 7fe2689d5700 1 -- 172.21.15.192:0/2544186641 wait complete. 2023-08-11T15:38:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2023-08-11T15:38:48.507 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-08-11T15:38:48.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph config generate-minimal-conf 2023-08-11T15:38:48.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:48 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:48.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:48 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:48.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:48 smithi032 ceph-mon[97103]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:48.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:48 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/2544186641' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T15:38:48.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:48 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:48.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:48 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:48.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:48 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:48.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:48 smithi192 ceph-mon[130806]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:48.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:48 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/2544186641' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T15:38:48.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:48 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:49.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:38:49.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: mon.c calling monitor election 2023-08-11T15:38:49.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: mon.a calling monitor election 2023-08-11T15:38:49.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:49.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:38:49.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:49.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:49.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:49.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:49.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:49.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:49.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:49.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T15:38:49.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:49.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: fsmap 2023-08-11T15:38:49.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:49.725 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: mgrmap e13: x(active, since 53s) 2023-08-11T15:38:49.725 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-11T15:38:49.725 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-11T15:38:49.725 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-11T15:38:49.726 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-08-11T15:38:49.726 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:49.726 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:49.726 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:49.727 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:49.727 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:49.727 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:49.728 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/2544186641' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T15:38:49.728 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: mon.b calling monitor election 2023-08-11T15:38:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Updating smithi032:/etc/ceph/ceph.conf 2023-08-11T15:38:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Updating smithi134:/etc/ceph/ceph.conf 2023-08-11T15:38:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:38:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: mon.b calling monitor election 2023-08-11T15:38:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: mon.c calling monitor election 2023-08-11T15:38:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: mon.a calling monitor election 2023-08-11T15:38:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-11T15:38:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: fsmap 2023-08-11T15:38:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: mgrmap e13: x(active, since 55s) 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Cluster is now healthy 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: overall HEALTH_OK 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:50 smithi032 ceph-mon[97103]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: mon.b calling monitor election 2023-08-11T15:38:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Updating smithi032:/etc/ceph/ceph.conf 2023-08-11T15:38:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Updating smithi134:/etc/ceph/ceph.conf 2023-08-11T15:38:50.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:38:50.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: mon.b calling monitor election 2023-08-11T15:38:50.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: mon.c calling monitor election 2023-08-11T15:38:50.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: mon.a calling monitor election 2023-08-11T15:38:50.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-11T15:38:50.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:50.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: fsmap 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: mgrmap e13: x(active, since 55s) 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Cluster is now healthy 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: overall HEALTH_OK 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:50 smithi192 ceph-mon[130806]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:50.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.704+0000 7f4258e22700 1 -- 172.21.15.32:0/1995381062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4254101570 msgr2=0x7f4254101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:50.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.704+0000 7f4258e22700 1 --2- 172.21.15.32:0/1995381062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4254101570 0x7f4254101950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4244009910 tx=0x7f4244009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:50.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.704+0000 7f4258e22700 1 -- 172.21.15.32:0/1995381062 shutdown_connections 2023-08-11T15:38:50.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.704+0000 7f4258e22700 1 --2- 172.21.15.32:0/1995381062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4254101570 0x7f4254101950 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:50.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.704+0000 7f4258e22700 1 -- 172.21.15.32:0/1995381062 >> 172.21.15.32:0/1995381062 conn(0x7f42540fade0 msgr2=0x7f42540fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:50.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.704+0000 7f4258e22700 1 -- 172.21.15.32:0/1995381062 shutdown_connections 2023-08-11T15:38:50.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.704+0000 7f4258e22700 1 -- 172.21.15.32:0/1995381062 wait complete. 2023-08-11T15:38:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 Processor -- start 2023-08-11T15:38:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 -- start start 2023-08-11T15:38:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4254101570 0x7f4254197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42541981f0 0x7f425419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f425419baf0 0x7f425419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42541988d0 con 0x7f42541981f0 2023-08-11T15:38:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42541a4940 con 0x7f4254101570 2023-08-11T15:38:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.706+0000 7f4258e22700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42541a4a80 con 0x7f425419baf0 2023-08-11T15:38:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42541981f0 0x7f425419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42541981f0 0x7f425419b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38288/0 (socket says 172.21.15.32:38288) 2023-08-11T15:38:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 -- 172.21.15.32:0/3647123784 learned_addr learned my addr 172.21.15.32:0/3647123784 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:38:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f425259c700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4254101570 0x7f4254197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4252d9d700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f425419baf0 0x7f425419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 -- 172.21.15.32:0/3647123784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f425419baf0 msgr2=0x7f425419bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f425419baf0 0x7f425419bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 -- 172.21.15.32:0/3647123784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4254101570 msgr2=0x7f4254197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:50.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4254101570 0x7f4254197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:50.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 -- 172.21.15.32:0/3647123784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440095f0 con 0x7f42541981f0 2023-08-11T15:38:50.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4252d9d700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f425419baf0 0x7f425419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:38:50.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.707+0000 7f4251d9b700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42541981f0 0x7f425419b5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f424800b780 tx=0x7f424800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:50.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.708+0000 7f42437fe700 1 -- 172.21.15.32:0/3647123784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248004230 con 0x7f42541981f0 2023-08-11T15:38:50.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.708+0000 7f42437fe700 1 -- 172.21.15.32:0/3647123784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4248004390 con 0x7f42541981f0 2023-08-11T15:38:50.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.708+0000 7f42437fe700 1 -- 172.21.15.32:0/3647123784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248011ad0 con 0x7f42541981f0 2023-08-11T15:38:50.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.708+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42541a4d30 con 0x7f42541981f0 2023-08-11T15:38:50.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.708+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42541a5180 con 0x7f42541981f0 2023-08-11T15:38:50.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.709+0000 7f42437fe700 1 -- 172.21.15.32:0/3647123784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f42480036a0 con 0x7f42541981f0 2023-08-11T15:38:50.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.709+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4254061f50 con 0x7f42541981f0 2023-08-11T15:38:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.710+0000 7f42437fe700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f423c038630 0x7f423c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.710+0000 7f42437fe700 1 -- 172.21.15.32:0/3647123784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f424804a4d0 con 0x7f42541981f0 2023-08-11T15:38:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.710+0000 7f425259c700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f423c038630 0x7f423c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.710+0000 7f425259c700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f423c038630 0x7f423c03aaf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4244006010 tx=0x7f424400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:50.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.715+0000 7f42437fe700 1 -- 172.21.15.32:0/3647123784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4248004500 con 0x7f42541981f0 2023-08-11T15:38:50.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: mon.b calling monitor election 2023-08-11T15:38:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Updating smithi032:/etc/ceph/ceph.conf 2023-08-11T15:38:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Updating smithi134:/etc/ceph/ceph.conf 2023-08-11T15:38:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:38:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: mon.b calling monitor election 2023-08-11T15:38:50.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: mon.c calling monitor election 2023-08-11T15:38:50.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: mon.a calling monitor election 2023-08-11T15:38:50.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-11T15:38:50.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} 2023-08-11T15:38:50.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: fsmap 2023-08-11T15:38:50.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: mgrmap e13: x(active, since 55s) 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Cluster is now healthy 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: overall HEALTH_OK 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:50.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:38:50.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:50 smithi134 ceph-mon[130447]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.940+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4254198fd0 con 0x7f42541981f0 2023-08-11T15:38:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.940+0000 7f42437fe700 1 -- 172.21.15.32:0/3647123784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f4248018e80 con 0x7f42541981f0 2023-08-11T15:38:50.942 INFO:teuthology.orchestra.run.smithi032.stdout:# minimal ceph.conf for cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:50.942 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2023-08-11T15:38:50.942 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:38:50.943 INFO:teuthology.orchestra.run.smithi032.stdout: mon_host = [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2023-08-11T15:38:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f423c038630 msgr2=0x7f423c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f423c038630 0x7f423c03aaf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4244006010 tx=0x7f424400b540 comp rx=0 tx=0).stop 2023-08-11T15:38:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42541981f0 msgr2=0x7f425419b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42541981f0 0x7f425419b5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f424800b780 tx=0x7f424800ba90 comp rx=0 tx=0).stop 2023-08-11T15:38:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 shutdown_connections 2023-08-11T15:38:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f423c038630 0x7f423c03aaf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f425419baf0 0x7f425419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4254101570 0x7f4254197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 --2- 172.21.15.32:0/3647123784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42541981f0 0x7f425419b5b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 >> 172.21.15.32:0/3647123784 conn(0x7f42540fade0 msgr2=0x7f4254106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 shutdown_connections 2023-08-11T15:38:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:38:50.943+0000 7f4258e22700 1 -- 172.21.15.32:0/3647123784 wait complete. 2023-08-11T15:38:51.437 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-08-11T15:38:51.437 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:38:51.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.conf 2023-08-11T15:38:51.512 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:38:51.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:51.629 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.630 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.630 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.630 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:51.630 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.630 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.630 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: Reconfiguring mon.a (unknown last config time)... 2023-08-11T15:38:51.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:51.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:51.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:51.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: Reconfiguring daemon mon.a on smithi032 2023-08-11T15:38:51.631 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:51 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3647123784' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:51.637 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:38:51.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2023-08-11T15:38:51.669 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.669 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.669 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.669 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: Reconfiguring mon.a (unknown last config time)... 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:51.670 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:51.671 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: Reconfiguring daemon mon.a on smithi032 2023-08-11T15:38:51.671 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:51 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3647123784' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:51.673 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:38:51.674 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:51.745 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:38:51.745 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2023-08-11T15:38:51.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:38:51.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:51.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: Reconfiguring mon.a (unknown last config time)... 2023-08-11T15:38:51.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:51.780 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:51.780 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:51.780 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: Reconfiguring daemon mon.a on smithi032 2023-08-11T15:38:51.780 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:51 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3647123784' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:51.783 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:38:51.783 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:38:51.853 INFO:tasks.cephadm:Adding mgr.x on smithi032 2023-08-11T15:38:51.853 INFO:tasks.cephadm:Adding mgr.y on smithi134 2023-08-11T15:38:51.854 INFO:tasks.cephadm:Adding mgr.z on smithi192 2023-08-11T15:38:51.854 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch apply mgr '3;smithi032=x;smithi134=y;smithi192=z' 2023-08-11T15:38:52.443 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:38:53.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:53.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:53.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:53.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: Reconfiguring mon.b (monmap changed)... 2023-08-11T15:38:53.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:53.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:53.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:53.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:53 smithi192 ceph-mon[130806]: Reconfiguring daemon mon.b on smithi134 2023-08-11T15:38:53.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: Reconfiguring mon.b (monmap changed)... 2023-08-11T15:38:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:53 smithi134 ceph-mon[130447]: Reconfiguring daemon mon.b on smithi134 2023-08-11T15:38:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: Reconfiguring mon.b (monmap changed)... 2023-08-11T15:38:53.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:53.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:53.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:53.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:53 smithi032 ceph-mon[97103]: Reconfiguring daemon mon.b on smithi134 2023-08-11T15:38:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 -- 172.21.15.192:0/422938183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68000ff440 msgr2=0x7f68000ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 --2- 172.21.15.192:0/422938183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68000ff440 0x7f68000ff860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009910 tx=0x7f67f8009c20 comp rx=0 tx=0).stop 2023-08-11T15:38:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 -- 172.21.15.192:0/422938183 shutdown_connections 2023-08-11T15:38:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 --2- 172.21.15.192:0/422938183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68000ff440 0x7f68000ff860 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 -- 172.21.15.192:0/422938183 >> 172.21.15.192:0/422938183 conn(0x7f68000fa9c0 msgr2=0x7f68000fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:53.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 -- 172.21.15.192:0/422938183 shutdown_connections 2023-08-11T15:38:53.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 -- 172.21.15.192:0/422938183 wait complete. 2023-08-11T15:38:53.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 Processor -- start 2023-08-11T15:38:53.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 -- start start 2023-08-11T15:38:53.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f68001957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:53.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.661+0000 7f680793c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6800195d00 0x7f680019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f680793c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6800196180 0x7f680019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f680793c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680019daa0 con 0x7f6800196180 2023-08-11T15:38:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f680793c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680019dc10 con 0x7f68000ff440 2023-08-11T15:38:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f680793c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680019ddd0 con 0x7f6800195d00 2023-08-11T15:38:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6800195d00 0x7f680019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6800195d00 0x7f680019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57408/0 (socket says 172.21.15.192:57408) 2023-08-11T15:38:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 -- 172.21.15.192:0/1198569333 learned_addr learned my addr 172.21.15.192:0/1198569333 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:38:53.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6805ed9700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6800196180 0x7f680019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:53.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f68056d8700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f68001957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:53.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 -- 172.21.15.192:0/1198569333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 msgr2=0x7f68001957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:53.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f68001957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 -- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6800196180 msgr2=0x7f680019d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:53.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6800196180 0x7f680019d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 -- 172.21.15.192:0/1198569333 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f80095f0 con 0x7f6800195d00 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f68056d8700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f68001957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6805ed9700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6800196180 0x7f680019d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f6804ed7700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6800195d00 0x7f680019ad70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67f400b780 tx=0x7f67f400bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f67fe7fc700 1 -- 172.21.15.192:0/1198569333 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4004230 con 0x7f6800195d00 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f67fe7fc700 1 -- 172.21.15.192:0/1198569333 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f4004390 con 0x7f6800195d00 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680019e0b0 con 0x7f6800195d00 2023-08-11T15:38:53.671 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f67fe7fc700 1 -- 172.21.15.192:0/1198569333 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4011b30 con 0x7f6800195d00 2023-08-11T15:38:53.672 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.662+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680019e630 con 0x7f6800195d00 2023-08-11T15:38:53.672 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.663+0000 7f67fe7fc700 1 -- 172.21.15.192:0/1198569333 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f67f4011cd0 con 0x7f6800195d00 2023-08-11T15:38:53.672 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.663+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680018ef60 con 0x7f6800195d00 2023-08-11T15:38:53.672 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.664+0000 7f67fe7fc700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f67e8038680 0x7f67e803ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:38:53.672 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.664+0000 7f67fe7fc700 1 -- 172.21.15.192:0/1198569333 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f67f404a760 con 0x7f6800195d00 2023-08-11T15:38:53.672 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.664+0000 7f68056d8700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f67e8038680 0x7f67e803ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:38:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.664+0000 7f68056d8700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f67e8038680 0x7f67e803ab40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009fd0 tx=0x7f67f80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:38:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.668+0000 7f67fe7fc700 1 -- 172.21.15.192:0/1198569333 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f67f401a400 con 0x7f6800195d00 2023-08-11T15:38:53.880 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.878+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi134=y;smithi192=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f680002ce50 con 0x7f67e8038680 2023-08-11T15:38:53.888 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.886+0000 7f67fe7fc700 1 -- 172.21.15.192:0/1198569333 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f680002ce50 con 0x7f67e8038680 2023-08-11T15:38:53.888 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2023-08-11T15:38:53.890 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f67e8038680 msgr2=0x7f67e803ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:53.890 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f67e8038680 0x7f67e803ab40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009fd0 tx=0x7f67f80058e0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.890 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6800195d00 msgr2=0x7f680019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:38:53.891 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6800195d00 0x7f680019ad70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67f400b780 tx=0x7f67f400bb40 comp rx=0 tx=0).stop 2023-08-11T15:38:53.891 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 shutdown_connections 2023-08-11T15:38:53.891 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f67e8038680 0x7f67e803ab40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.891 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6800196180 0x7f680019d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.891 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f68001957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.891 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 --2- 172.21.15.192:0/1198569333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6800195d00 0x7f680019ad70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:38:53.891 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.888+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 >> 172.21.15.192:0/1198569333 conn(0x7f68000fa9c0 msgr2=0x7f68000691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:38:53.892 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.889+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 shutdown_connections 2023-08-11T15:38:53.892 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:38:53.889+0000 7f680793c700 1 -- 172.21.15.192:0/1198569333 wait complete. 2023-08-11T15:38:54.385 DEBUG:teuthology.orchestra.run.smithi134:mgr.y> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.y.service 2023-08-11T15:38:54.388 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.z.service 2023-08-11T15:38:54.391 INFO:tasks.cephadm:Deploying OSDs... 2023-08-11T15:38:54.391 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:38:54.392 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T15:38:54.421 INFO:journalctl@ceph.mgr.z.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:38:54.422 INFO:journalctl@ceph.mgr.y.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:38:54.441 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T15:38:54.442 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2023-08-11T15:38:54.491 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T15:38:54.491 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:54.491 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-08-11T15:38:54.491 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:54.491 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:54.492 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-08-11 15:38:03.092138000 +0000 2023-08-11T15:38:54.492 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-08-11 15:31:50.849533052 +0000 2023-08-11T15:38:54.492 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-08-11 15:31:50.849533052 +0000 2023-08-11T15:38:54.492 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-08-11 15:31:50.825533531 +0000 2023-08-11T15:38:54.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T15:38:54.561 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-08-11T15:38:54.561 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-08-11T15:38:54.561 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000191525 s, 2.7 MB/s 2023-08-11T15:38:54.564 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T15:38:54.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: Reconfiguring mon.c (monmap changed)... 2023-08-11T15:38:54.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:54.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:54.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:54.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: Reconfiguring daemon mon.c on smithi192 2023-08-11T15:38:54.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi134=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:54.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: Saving service mgr spec with placement smithi032=x;smithi134=y;smithi192=z;count:3 2023-08-11T15:38:54.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:54 smithi192 ceph-mon[130806]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:54.613 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2023-08-11T15:38:54.699 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T15:38:54.700 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:54.700 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-08-11T15:38:54.700 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:54.700 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:54.700 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-08-11 15:38:03.092138000 +0000 2023-08-11T15:38:54.700 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-08-11 15:31:51.683516414 +0000 2023-08-11T15:38:54.700 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-08-11 15:31:51.683516414 +0000 2023-08-11T15:38:54.701 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-08-11 15:31:51.661516852 +0000 2023-08-11T15:38:54.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T15:38:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: Reconfiguring mon.c (monmap changed)... 2023-08-11T15:38:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: Reconfiguring daemon mon.c on smithi192 2023-08-11T15:38:54.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi134=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:54.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: Saving service mgr spec with placement smithi032=x;smithi134=y;smithi192=z;count:3 2023-08-11T15:38:54.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:54 smithi134 ceph-mon[130447]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:54.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.800 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.800 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: Reconfiguring mon.c (monmap changed)... 2023-08-11T15:38:54.800 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T15:38:54.800 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T15:38:54.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:54.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: Reconfiguring daemon mon.c on smithi192 2023-08-11T15:38:54.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi134=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:38:54.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: Saving service mgr spec with placement smithi032=x;smithi134=y;smithi192=z;count:3 2023-08-11T15:38:54.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:54.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:54 smithi032 ceph-mon[97103]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:54.807 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-08-11T15:38:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-08-11T15:38:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000187504 s, 2.7 MB/s 2023-08-11T15:38:54.809 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T15:38:54.854 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2023-08-11T15:38:54.939 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T15:38:54.939 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:54.940 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-08-11T15:38:54.940 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:54.940 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:54.940 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-08-11 15:38:03.092138000 +0000 2023-08-11T15:38:54.940 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-08-11 15:31:52.510499915 +0000 2023-08-11T15:38:54.941 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-08-11 15:31:52.510499915 +0000 2023-08-11T15:38:54.941 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-08-11 15:31:52.489500334 +0000 2023-08-11T15:38:54.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T15:38:55.045 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-08-11T15:38:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-08-11T15:38:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000194592 s, 2.6 MB/s 2023-08-11T15:38:55.049 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T15:38:55.096 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2023-08-11T15:38:55.180 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T15:38:55.181 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:55.181 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-08-11T15:38:55.181 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:55.182 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:55.182 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-08-11 15:38:03.092138000 +0000 2023-08-11T15:38:55.183 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-08-11 15:31:53.348483197 +0000 2023-08-11T15:38:55.183 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-08-11 15:31:53.348483197 +0000 2023-08-11T15:38:55.183 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-08-11 15:31:53.326483635 +0000 2023-08-11T15:38:55.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T15:38:55.286 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-08-11T15:38:55.286 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-08-11T15:38:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000198903 s, 2.6 MB/s 2023-08-11T15:38:55.289 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T15:38:55.336 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-08-11T15:38:55.337 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T15:38:55.358 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T15:38:55.358 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2023-08-11T15:38:55.419 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T15:38:55.419 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:55.420 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 377777 Links: 1 2023-08-11T15:38:55.420 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:55.420 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:55.421 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-08-11 15:38:41.819254492 +0000 2023-08-11T15:38:55.421 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-08-11 15:27:49.431458583 +0000 2023-08-11T15:38:55.421 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-08-11 15:27:49.431458583 +0000 2023-08-11T15:38:55.421 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-08-11T15:38:55.421 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T15:38:55.491 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-08-11T15:38:55.491 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-08-11T15:38:55.491 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000246366 s, 2.1 MB/s 2023-08-11T15:38:55.492 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T15:38:55.552 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2023-08-11T15:38:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T15:38:55.612 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:55.612 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 386356 Links: 1 2023-08-11T15:38:55.612 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:55.613 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:55.613 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-08-11 15:38:41.819254492 +0000 2023-08-11T15:38:55.613 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-08-11 15:27:50.122442764 +0000 2023-08-11T15:38:55.613 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-08-11 15:27:50.122442764 +0000 2023-08-11T15:38:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-08-11T15:38:55.614 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T15:38:55.680 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-08-11T15:38:55.680 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-08-11T15:38:55.681 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000211893 s, 2.4 MB/s 2023-08-11T15:38:55.682 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T15:38:55.748 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2023-08-11T15:38:55.807 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T15:38:55.808 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:55.808 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 386480 Links: 1 2023-08-11T15:38:55.808 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:55.809 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:55.809 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-08-11 15:38:41.819254492 +0000 2023-08-11T15:38:55.809 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-08-11 15:27:50.664430355 +0000 2023-08-11T15:38:55.810 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-08-11 15:27:50.664430355 +0000 2023-08-11T15:38:55.810 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-08-11T15:38:55.811 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T15:38:55.873 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:55.873 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:55.874 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:55.874 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:55.874 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:55.874 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:55.874 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:38:55.874 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T15:38:55.874 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:38:55.875 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:55.875 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:55 smithi134 ceph-mon[130447]: Deploying daemon mgr.z on smithi192 2023-08-11T15:38:55.876 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-08-11T15:38:55.876 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-08-11T15:38:55.876 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000213855 s, 2.4 MB/s 2023-08-11T15:38:55.877 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T15:38:55.936 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2023-08-11T15:38:55.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:55.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:55.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:55.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:55.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:55.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:55.998 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:38:55.998 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T15:38:55.998 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:38:55.998 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:55.998 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:55 smithi192 ceph-mon[130806]: Deploying daemon mgr.z on smithi192 2023-08-11T15:38:55.999 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T15:38:55.999 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:55.999 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 386581 Links: 1 2023-08-11T15:38:56.000 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:56.000 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:56.001 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-08-11 15:38:41.819254492 +0000 2023-08-11T15:38:56.001 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-08-11 15:27:51.192418267 +0000 2023-08-11T15:38:56.001 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-08-11 15:27:51.192418267 +0000 2023-08-11T15:38:56.002 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-08-11T15:38:56.002 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T15:38:56.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:56.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:56.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:38:56.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:56.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:38:56.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:56.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:38:56.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T15:38:56.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:38:56.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:56.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:55 smithi032 ceph-mon[97103]: Deploying daemon mgr.z on smithi192 2023-08-11T15:38:56.067 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-08-11T15:38:56.068 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-08-11T15:38:56.068 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00019897 s, 2.6 MB/s 2023-08-11T15:38:56.068 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T15:38:56.126 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-08-11T15:38:56.126 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T15:38:56.152 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T15:38:56.152 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2023-08-11T15:38:56.210 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T15:38:56.211 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:56.211 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 376827 Links: 1 2023-08-11T15:38:56.212 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:56.212 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:56.212 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-08-11 15:38:56.165635817 +0000 2023-08-11T15:38:56.213 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-08-11 15:27:49.732849276 +0000 2023-08-11T15:38:56.213 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-08-11 15:27:49.732849276 +0000 2023-08-11T15:38:56.213 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-08-11T15:38:56.214 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T15:38:56.279 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:56 smithi192 systemd[1]: Starting Ceph mgr.z for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:38:56.283 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-08-11T15:38:56.284 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-08-11T15:38:56.284 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000232533 s, 2.2 MB/s 2023-08-11T15:38:56.285 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T15:38:56.344 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2023-08-11T15:38:56.403 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T15:38:56.403 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:56.404 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 390218 Links: 1 2023-08-11T15:38:56.404 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:56.404 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:56.404 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-08-11 15:38:56.165635817 +0000 2023-08-11T15:38:56.404 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-08-11 15:27:50.259839000 +0000 2023-08-11T15:38:56.404 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-08-11 15:27:50.259839000 +0000 2023-08-11T15:38:56.404 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-08-11T15:38:56.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T15:38:56.468 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-08-11T15:38:56.469 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-08-11T15:38:56.469 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000251916 s, 2.0 MB/s 2023-08-11T15:38:56.470 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T15:38:56.529 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2023-08-11T15:38:56.583 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:56 smithi192 podman[132726]: 2023-08-11T15:38:56.587 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T15:38:56.587 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:56.587 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 390272 Links: 1 2023-08-11T15:38:56.587 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:56.587 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:56.588 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-08-11 15:38:56.165635817 +0000 2023-08-11T15:38:56.588 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-08-11 15:27:50.773828976 +0000 2023-08-11T15:38:56.588 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-08-11 15:27:50.773828976 +0000 2023-08-11T15:38:56.588 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-08-11T15:38:56.588 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T15:38:56.653 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-08-11T15:38:56.654 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-08-11T15:38:56.654 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000209347 s, 2.4 MB/s 2023-08-11T15:38:56.655 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T15:38:56.713 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2023-08-11T15:38:56.770 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T15:38:56.770 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T15:38:56.770 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 385702 Links: 1 2023-08-11T15:38:56.771 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:38:56.771 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T15:38:56.771 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-08-11 15:38:56.165635817 +0000 2023-08-11T15:38:56.771 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-08-11 15:27:51.304818621 +0000 2023-08-11T15:38:56.771 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-08-11 15:27:51.304818621 +0000 2023-08-11T15:38:56.771 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-08-11T15:38:56.772 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T15:38:56.837 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-08-11T15:38:56.837 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-08-11T15:38:56.837 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000180571 s, 2.8 MB/s 2023-08-11T15:38:56.838 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T15:38:56.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:56 smithi192 ceph-mon[130806]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:56.881 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:56 smithi192 bash[132726]: 61c16becd479c5862f112242fd6a021b873469bb34b29b0e2f3719b2614b1a4a 2023-08-11T15:38:56.881 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:56 smithi192 ceph-mgr[132890]: -- 172.21.15.192:0/1113780427 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56354d3791e0 con 0x56354d34f000 2023-08-11T15:38:56.882 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:56 smithi192 systemd[1]: Started Ceph mgr.z for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:38:56.899 INFO:tasks.cephadm:Deploying osd.0 on smithi032 with /dev/vg_nvme/lv_4... 2023-08-11T15:38:56.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-11T15:38:56.967 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:56 smithi032 ceph-mon[97103]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:56.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:56 smithi134 ceph-mon[130447]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:57.142 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:56 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:56.877+0000 7f5b09b99200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:38:57.143 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:56 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:56.961+0000 7f5b09b99200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:38:57.143 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:57.048+0000 7f5b09b99200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:38:57.583 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:57.141+0000 7f5b09b99200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:38:57.583 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:57.308+0000 7f5b09b99200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:38:57.635 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:38:57.982 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:57.671+0000 7f5b09b99200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:38:57.983 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:57.800+0000 7f5b09b99200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:38:57.983 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:57.887+0000 7f5b09b99200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:38:57.983 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:57.981+0000 7f5b09b99200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:38:57.983 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:57.984 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:57.984 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:57.984 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:38:57.984 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T15:38:57.984 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:38:57.984 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:57.984 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:57 smithi192 ceph-mon[130806]: Deploying daemon mgr.y on smithi134 2023-08-11T15:38:58.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:58.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:58.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:58.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:38:58.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T15:38:58.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:38:58.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:58.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:57 smithi134 ceph-mon[130447]: Deploying daemon mgr.y on smithi134 2023-08-11T15:38:58.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:58.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:58.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:38:58.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:38:58.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T15:38:58.202 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:38:58.202 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:38:58.202 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:57 smithi032 ceph-mon[97103]: Deploying daemon mgr.y on smithi134 2023-08-11T15:38:58.470 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:58 smithi134 systemd[1]: Starting Ceph mgr.y for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:38:58.833 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:58 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:58.408+0000 7f5b09b99200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:38:58.879 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:58 smithi134 podman[131881]: 2023-08-11T15:38:59.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:58 smithi032 ceph-mon[97103]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:59.155 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:58 smithi192 ceph-mon[130806]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:59.155 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:59.072+0000 7f5b09b99200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:38:59.174 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:58 smithi134 ceph-mon[130447]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:38:59.175 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:58 smithi134 bash[131881]: 2387ae3fb3e6cb788abad511e99fc1c9bb342e2b5edfc14dc62024d43eb33f24 2023-08-11T15:38:59.175 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:58 smithi134 ceph-mgr[131922]: -- 172.21.15.134:0/3321925523 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55803a8551e0 con 0x55803a82a800 2023-08-11T15:38:59.175 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:58 smithi134 systemd[1]: Started Ceph mgr.y for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:38:59.176 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:38:59.002+0000 7f0d53397200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:38:59.176 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:38:59.082+0000 7f0d53397200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:38:59.177 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:38:59.173+0000 7f0d53397200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:38:59.452 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:38:59.272+0000 7f0d53397200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:38:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-11T15:38:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-11T15:38:59.480 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-08-11T15:38:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-08-11T15:38:59.480 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0331147 s, 317 MB/s 2023-08-11T15:38:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-08-11T15:38:59.583 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:59.153+0000 7f5b09b99200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:38:59.720 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:38:59.451+0000 7f0d53397200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:38:59.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi032:vg_nvme/lv_4 2023-08-11T15:39:00.171 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:38:59.825+0000 7f0d53397200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:39:00.171 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:38:59.963+0000 7f0d53397200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:39:00.171 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:00.064+0000 7f0d53397200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:39:00.172 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.172 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.172 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.172 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.172 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.172 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:38:59 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:00.283 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:38:59.998+0000 7f5b09b99200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:39:00.283 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:00.090+0000 7f5b09b99200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:39:00.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.284 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.284 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.284 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:38:59 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:00.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:59 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:59 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:59 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:59 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:59 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:00.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:38:59 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:39:00.833 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:00.280+0000 7f5b09b99200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:39:00.833 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:00.576+0000 7f5b09b99200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:39:00.834 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:00.808+0000 7f5b09b99200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:39:00.970 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:00.168+0000 7f0d53397200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:39:00.971 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:00.615+0000 7f0d53397200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:39:01.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:01.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:01.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:01.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:39:01.182 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:39:01.182 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:01.225 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:00.961+0000 7f5b09b99200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:39:01.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:01.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.227 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.227 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:01.227 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:01.227 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.227 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:39:01.227 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:39:01.227 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:01.289 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:01.289 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.290 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.290 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.290 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.290 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:01.290 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:01.291 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:01.291 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T15:39:01.291 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T15:39:01.291 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:01.583 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:01.224+0000 7f5b09b99200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:39:01.720 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:01.286+0000 7f0d53397200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:39:01.721 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:01.367+0000 7f0d53397200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:39:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.211+0000 7f843d86c700 1 -- 172.21.15.32:0/363576741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438073380 msgr2=0x7f8438073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.211+0000 7f843d86c700 1 --2- 172.21.15.32:0/363576741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438073380 0x7f8438073800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f842c009990 tx=0x7f842c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:39:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.211+0000 7f843d86c700 1 -- 172.21.15.32:0/363576741 shutdown_connections 2023-08-11T15:39:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.211+0000 7f843d86c700 1 --2- 172.21.15.32:0/363576741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438074950 0x7f8438072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.211+0000 7f843d86c700 1 --2- 172.21.15.32:0/363576741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438108880 0x7f843810ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.211+0000 7f843d86c700 1 --2- 172.21.15.32:0/363576741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438073380 0x7f8438073800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.211+0000 7f843d86c700 1 -- 172.21.15.32:0/363576741 >> 172.21.15.32:0/363576741 conn(0x7f84380fbf10 msgr2=0x7f84380fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:39:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.212+0000 7f843d86c700 1 -- 172.21.15.32:0/363576741 shutdown_connections 2023-08-11T15:39:02.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.212+0000 7f843d86c700 1 -- 172.21.15.32:0/363576741 wait complete. 2023-08-11T15:39:02.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 Processor -- start 2023-08-11T15:39:02.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 -- start start 2023-08-11T15:39:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438073380 0x7f84381988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438074950 0x7f8438198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438108880 0x7f843819fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84381a0460 con 0x7f8438074950 2023-08-11T15:39:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84381a05a0 con 0x7f8438108880 2023-08-11T15:39:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f843d86c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84381a0730 con 0x7f8438073380 2023-08-11T15:39:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f8436ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438073380 0x7f84381988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.213+0000 7f8436ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438073380 0x7f84381988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:33510/0 (socket says 172.21.15.32:33510) 2023-08-11T15:39:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f8436ffd700 1 -- 172.21.15.32:0/4230267758 learned_addr learned my addr 172.21.15.32:0/4230267758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:39:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f84367fc700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438074950 0x7f8438198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f84377fe700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438108880 0x7f843819fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f8436ffd700 1 -- 172.21.15.32:0/4230267758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438108880 msgr2=0x7f843819fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f8436ffd700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438108880 0x7f843819fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f8436ffd700 1 -- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438074950 msgr2=0x7f8438198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:02.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f8436ffd700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438074950 0x7f8438198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:02.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f8436ffd700 1 -- 172.21.15.32:0/4230267758 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0095f0 con 0x7f8438073380 2023-08-11T15:39:02.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f8436ffd700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438073380 0x7f84381988d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9a0 tx=0x7f842800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:39:02.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428009790 con 0x7f8438073380 2023-08-11T15:39:02.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842800c810 con 0x7f8438073380 2023-08-11T15:39:02.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.214+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84381a0a10 con 0x7f8438073380 2023-08-11T15:39:02.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.215+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84280054c0 con 0x7f8438073380 2023-08-11T15:39:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.215+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84381a0f60 con 0x7f8438073380 2023-08-11T15:39:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.216+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) v1 ==== 45191+0+0 (secure 0 0 0) 0x7f8428021440 con 0x7f8438073380 2023-08-11T15:39:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.216+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8438106f30 con 0x7f8438073380 2023-08-11T15:39:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.216+0000 7f843c86a700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f84200383e0 0x7f842003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.216+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8428014070 con 0x7f8438073380 2023-08-11T15:39:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.216+0000 7f84367fc700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f84200383e0 0x7f842003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.217+0000 7f84367fc700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f84200383e0 0x7f842003a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b680 tx=0x7f842c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:39:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.221+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8428005660 con 0x7f8438073380 2023-08-11T15:39:02.247 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:01 smithi192 ceph-mon[130806]: Reconfiguring mgr.x (unknown last config time)... 2023-08-11T15:39:02.248 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:01 smithi192 ceph-mon[130806]: Reconfiguring daemon mgr.x on smithi032 2023-08-11T15:39:02.248 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:01.922+0000 7f5b09b99200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:39:02.248 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:02.052+0000 7f5b09b99200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:39:02.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:01 smithi032 ceph-mon[97103]: Reconfiguring mgr.x (unknown last config time)... 2023-08-11T15:39:02.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:01 smithi032 ceph-mon[97103]: Reconfiguring daemon mgr.x on smithi032 2023-08-11T15:39:02.388 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:01 smithi134 ceph-mon[130447]: Reconfiguring mgr.x (unknown last config time)... 2023-08-11T15:39:02.388 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:01 smithi134 ceph-mon[130447]: Reconfiguring daemon mgr.x on smithi032 2023-08-11T15:39:02.388 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:02.127+0000 7f0d53397200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:39:02.388 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:02.216+0000 7f0d53397200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:39:02.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:02.452+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f843802ce30 con 0x7f84200383e0 2023-08-11T15:39:02.576 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:02.246+0000 7f5b09b99200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:39:02.576 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:02.357+0000 7f5b09b99200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:39:02.663 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:02.386+0000 7f0d53397200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:39:02.833 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:02.574+0000 7f5b09b99200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:39:02.970 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:02.662+0000 7f0d53397200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:39:02.971 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:02.890+0000 7f0d53397200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:39:03.298 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:03.039+0000 7f0d53397200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:39:03.299 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:03.299 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:03.299 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:03.300 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:03.300 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.300 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.300 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:03.301 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:03.302 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:03.302 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:03.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:03.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:03.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:03.720 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:03.297+0000 7f0d53397200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:39:03.834 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:39:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:39:03.436+0000 7f5b09b99200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:39:04.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:03.999+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mgrmap(e 14) v1 ==== 89903+0+0 (secure 0 0 0) 0x7f842800cc80 con 0x7f8438073380 2023-08-11T15:39:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:03 smithi032 ceph-mon[97103]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:03 smithi032 ceph-mon[97103]: Standby manager daemon z started 2023-08-11T15:39:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:03 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T15:39:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:03 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:39:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:03 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T15:39:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:03 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:39:04.307 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:03.988+0000 7f0d53397200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:39:04.307 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:04 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:04.117+0000 7f0d53397200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:39:04.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-mon[130447]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:04.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-mon[130447]: Standby manager daemon z started 2023-08-11T15:39:04.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T15:39:04.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:39:04.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T15:39:04.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:03 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:39:04.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:03 smithi192 ceph-mon[130806]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:04.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:03 smithi192 ceph-mon[130806]: Standby manager daemon z started 2023-08-11T15:39:04.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:03 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T15:39:04.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:03 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:39:04.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:03 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T15:39:04.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:03 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3970404868' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:39:04.631 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:04 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:04.305+0000 7f0d53397200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:39:04.631 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:04 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:04.416+0000 7f0d53397200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:39:04.970 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:04 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:04.629+0000 7f0d53397200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:39:05.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:05 smithi032 ceph-mon[97103]: mgrmap e14: x(active, since 70s), standbys: z 2023-08-11T15:39:05.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T15:39:05.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:05 smithi032 ceph-mon[97103]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:05.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:05 smithi192 ceph-mon[130806]: mgrmap e14: x(active, since 70s), standbys: z 2023-08-11T15:39:05.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T15:39:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:05 smithi192 ceph-mon[130806]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:05.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:05 smithi134 ceph-mon[130447]: mgrmap e14: x(active, since 70s), standbys: z 2023-08-11T15:39:05.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T15:39:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:05 smithi134 ceph-mon[130447]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:05.970 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:39:05 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:39:05.477+0000 7f0d53397200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:39:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:06.015+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mon.1 v2:172.21.15.192:3300/0 8 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7f842800c980 con 0x7f8438073380 2023-08-11T15:39:06.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: Standby manager daemon y started 2023-08-11T15:39:06.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T15:39:06.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:39:06.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T15:39:06.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:39:06.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1291112541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]: dispatch 2023-08-11T15:39:06.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]: dispatch 2023-08-11T15:39:06.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]': finished 2023-08-11T15:39:06.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: osdmap e5: 1 total, 0 up, 1 in 2023-08-11T15:39:06.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:06.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: Standby manager daemon y started 2023-08-11T15:39:06.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T15:39:06.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:39:06.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T15:39:06.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:39:06.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1291112541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]: dispatch 2023-08-11T15:39:06.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]: dispatch 2023-08-11T15:39:06.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]': finished 2023-08-11T15:39:06.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: osdmap e5: 1 total, 0 up, 1 in 2023-08-11T15:39:06.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:06.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: Standby manager daemon y started 2023-08-11T15:39:06.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T15:39:06.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:39:06.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T15:39:06.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/1819999476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:39:06.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1291112541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]: dispatch 2023-08-11T15:39:06.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]: dispatch 2023-08-11T15:39:06.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "880cacc2-d87f-43c6-902e-9dfbab59b3c6"}]': finished 2023-08-11T15:39:06.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: osdmap e5: 1 total, 0 up, 1 in 2023-08-11T15:39:06.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:07 smithi032 ceph-mon[97103]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:07 smithi032 ceph-mon[97103]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-08-11T15:39:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T15:39:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:07 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2338340857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:07.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:07 smithi192 ceph-mon[130806]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:07.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:07 smithi192 ceph-mon[130806]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-08-11T15:39:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T15:39:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:07 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2338340857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:07.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:07 smithi134 ceph-mon[130447]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:07 smithi134 ceph-mon[130447]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-08-11T15:39:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T15:39:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:07 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2338340857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:09.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:09 smithi032 ceph-mon[97103]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:09.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:09 smithi192 ceph-mon[130806]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:09.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:09 smithi134 ceph-mon[130447]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:11.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:11 smithi032 ceph-mon[97103]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:11.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:11 smithi192 ceph-mon[130806]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:11.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:11 smithi134 ceph-mon[130447]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:12.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:12 smithi032 ceph-mon[97103]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:13.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:12 smithi192 ceph-mon[130806]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:13.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:12 smithi134 ceph-mon[130447]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:14.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-11T15:39:14.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:14.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:13 smithi032 ceph-mon[97103]: Deploying daemon osd.0 on smithi032 2023-08-11T15:39:14.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-11T15:39:14.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:14.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:13 smithi192 ceph-mon[130806]: Deploying daemon osd.0 on smithi032 2023-08-11T15:39:14.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-11T15:39:14.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:14.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:13 smithi134 ceph-mon[130447]: Deploying daemon osd.0 on smithi032 2023-08-11T15:39:15.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:14 smithi032 ceph-mon[97103]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:15.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:14 smithi192 ceph-mon[130806]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:15.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:14 smithi134 ceph-mon[130447]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:17.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:17 smithi192 ceph-mon[130806]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:17.339 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:17 smithi032 ceph-mon[97103]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:17.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:17 smithi134 ceph-mon[130447]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:19.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:19 smithi032 ceph-mon[97103]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:19.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:19 smithi192 ceph-mon[130806]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:19.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:19 smithi134 ceph-mon[130447]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:20.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:20.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:20.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:20.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:20 smithi032 ceph-mon[97103]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:20.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:20.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:20.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:20 smithi192 ceph-mon[130806]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:20.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:20.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:20.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:20.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:20 smithi134 ceph-mon[130447]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:22.867 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 0 on host 'smithi032' 2023-08-11T15:39:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.866+0000 7f843c86a700 1 -- 172.21.15.32:0/4230267758 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f843802ce30 con 0x7f84200383e0 2023-08-11T15:39:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f84200383e0 msgr2=0x7f842003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f84200383e0 0x7f842003a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b680 tx=0x7f842c005fd0 comp rx=0 tx=0).stop 2023-08-11T15:39:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438073380 msgr2=0x7f84381988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438073380 0x7f84381988d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9a0 tx=0x7f842800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:39:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 shutdown_connections 2023-08-11T15:39:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f84200383e0 0x7f842003a8a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8438108880 0x7f843819fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8438073380 0x7f84381988d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 --2- 172.21.15.32:0/4230267758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438074950 0x7f8438198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 >> 172.21.15.32:0/4230267758 conn(0x7f84380fbf10 msgr2=0x7f84380fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:39:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 shutdown_connections 2023-08-11T15:39:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:39:22.868+0000 7f843d86c700 1 -- 172.21.15.32:0/4230267758 wait complete. 2023-08-11T15:39:22.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:22.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:22.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:22.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:22.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:22.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:22 smithi134 ceph-mon[130447]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:23.006 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:23.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:23.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:23.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:23.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:23.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:22 smithi032 ceph-mon[97103]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:23.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:22 smithi192 ceph-mon[130806]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:23.338 DEBUG:teuthology.orchestra.run.smithi032:osd.0> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.0.service 2023-08-11T15:39:23.343 INFO:tasks.cephadm:Deploying osd.1 on smithi134 with /dev/vg_nvme/lv_4... 2023-08-11T15:39:23.343 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-11T15:39:23.415 INFO:journalctl@ceph.osd.0.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:39:23.907 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:39:24.155 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:24.156 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:24.156 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:24.156 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:23 smithi032 ceph-mon[97103]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T15:39:24.167 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:23 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:24.167 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:23 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:24.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:23 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:24.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:23 smithi134 ceph-mon[130447]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T15:39:24.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:23 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:23 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:23 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:23 smithi192 ceph-mon[130806]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T15:39:25.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:24 smithi134 ceph-mon[130447]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-11T15:39:25.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:24 smithi134 ceph-mon[130447]: osdmap e6: 1 total, 0 up, 1 in 2023-08-11T15:39:25.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:24 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:25.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:24 smithi134 ceph-mon[130447]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:39:25.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:24 smithi134 ceph-mon[130447]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:25.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:24 smithi032 ceph-mon[97103]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-11T15:39:25.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:24 smithi032 ceph-mon[97103]: osdmap e6: 1 total, 0 up, 1 in 2023-08-11T15:39:25.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:24 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:25.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:24 smithi032 ceph-mon[97103]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:39:25.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:24 smithi032 ceph-mon[97103]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:25.304 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:39:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-0[109333]: 2023-08-11T15:39:24.853+0000 7f285e82a700 -1 osd.0 0 waiting for initial osdmap 2023-08-11T15:39:25.304 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:39:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-0[109333]: 2023-08-11T15:39:24.857+0000 7f285861b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:39:25.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:24 smithi192 ceph-mon[130806]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-11T15:39:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:24 smithi192 ceph-mon[130806]: osdmap e6: 1 total, 0 up, 1 in 2023-08-11T15:39:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:24 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:24 smithi192 ceph-mon[130806]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:39:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:24 smithi192 ceph-mon[130806]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:25.380 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-11T15:39:25.381 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-11T15:39:25.381 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-08-11T15:39:25.381 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-08-11T15:39:25.381 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109691 s, 956 MB/s 2023-08-11T15:39:25.381 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-08-11T15:39:25.849 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2023-08-11T15:39:26.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:25 smithi032 ceph-mon[97103]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:39:26.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:25 smithi032 ceph-mon[97103]: osdmap e7: 1 total, 0 up, 1 in 2023-08-11T15:39:26.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:25 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:26.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:25 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:26.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:25 smithi134 ceph-mon[130447]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:39:26.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:25 smithi134 ceph-mon[130447]: osdmap e7: 1 total, 0 up, 1 in 2023-08-11T15:39:26.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:25 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:26.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:25 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:26.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:25 smithi192 ceph-mon[130806]: from='osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:39:26.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:25 smithi192 ceph-mon[130806]: osdmap e7: 1 total, 0 up, 1 in 2023-08-11T15:39:26.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:25 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:26.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:25 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:26.398 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:39:27.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:39:27.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:39:27.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] boot 2023-08-11T15:39:27.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: osdmap e8: 1 total, 1 up, 1 in 2023-08-11T15:39:27.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:27.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:27.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi032 2023-08-11T15:39:27.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:39:27.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi032 to 17272M 2023-08-11T15:39:27.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:27.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:27.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:39:27.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:39:27.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] boot 2023-08-11T15:39:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: osdmap e8: 1 total, 1 up, 1 in 2023-08-11T15:39:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi032 2023-08-11T15:39:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:39:27.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi032 to 17272M 2023-08-11T15:39:27.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:27.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:27.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:39:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:39:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: osd.0 [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] boot 2023-08-11T15:39:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: osdmap e8: 1 total, 1 up, 1 in 2023-08-11T15:39:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:39:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T15:39:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi032 2023-08-11T15:39:27.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:39:27.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi032 to 17272M 2023-08-11T15:39:27.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:27.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:27.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:27.599 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 -- 172.21.15.134:0/2251909606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 msgr2=0x7f13681006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:27.599 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2251909606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f13681006d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13540099c0 tx=0x7f1354009cd0 comp rx=0 tx=0).stop 2023-08-11T15:39:27.599 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 -- 172.21.15.134:0/2251909606 shutdown_connections 2023-08-11T15:39:27.599 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2251909606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13680ff070 0x7f13680ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:27.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2251909606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1368100c10 0x7f13681074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:27.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2251909606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f13681006d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:27.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 -- 172.21.15.134:0/2251909606 >> 172.21.15.134:0/2251909606 conn(0x7f13680fa610 msgr2=0x7f13680fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:39:27.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 -- 172.21.15.134:0/2251909606 shutdown_connections 2023-08-11T15:39:27.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 -- 172.21.15.134:0/2251909606 wait complete. 2023-08-11T15:39:27.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 Processor -- start 2023-08-11T15:39:27.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.596+0000 7f136ebbc700 1 -- start start 2023-08-11T15:39:27.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136ebbc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13680ff070 0x7f13681966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:27.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136ebbc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f1368196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:27.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136ebbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1368100c10 0x7f136819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:27.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136ebbc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819e260 con 0x7f1368100c10 2023-08-11T15:39:27.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136ebbc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819e3a0 con 0x7f1368100270 2023-08-11T15:39:27.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136ebbc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819e530 con 0x7f13680ff070 2023-08-11T15:39:27.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f1368196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f1368196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53240/0 (socket says 172.21.15.134:53240) 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 -- 172.21.15.134:0/2196555413 learned_addr learned my addr 172.21.15.134:0/2196555413 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136c958700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13680ff070 0x7f13681966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136d159700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1368100c10 0x7f136819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 -- 172.21.15.134:0/2196555413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13680ff070 msgr2=0x7f13681966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13680ff070 0x7f13681966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 -- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1368100c10 msgr2=0x7f136819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:27.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1368100c10 0x7f136819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:27.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 -- 172.21.15.134:0/2196555413 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13540095f0 con 0x7f1368100270 2023-08-11T15:39:27.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1367fff700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f1368196c10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1354009990 tx=0x7f1354004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:39:27.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1365ffb700 1 -- 172.21.15.134:0/2196555413 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135401d070 con 0x7f1368100270 2023-08-11T15:39:27.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136819e7b0 con 0x7f1368100270 2023-08-11T15:39:27.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1365ffb700 1 -- 172.21.15.134:0/2196555413 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1354004090 con 0x7f1368100270 2023-08-11T15:39:27.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.597+0000 7f1365ffb700 1 -- 172.21.15.134:0/2196555413 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354004a00 con 0x7f1368100270 2023-08-11T15:39:27.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.598+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136819ec50 con 0x7f1368100270 2023-08-11T15:39:27.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.599+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136810b100 con 0x7f1368100270 2023-08-11T15:39:27.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.600+0000 7f1365ffb700 1 -- 172.21.15.134:0/2196555413 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7f1354003bd0 con 0x7f1368100270 2023-08-11T15:39:27.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.600+0000 7f1365ffb700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f135809fec0 0x7f13580a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:27.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.600+0000 7f1365ffb700 1 -- 172.21.15.134:0/2196555413 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f13540ca730 con 0x7f1368100270 2023-08-11T15:39:27.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.601+0000 7f136c958700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f135809fec0 0x7f13580a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:27.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.601+0000 7f136c958700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f135809fec0 0x7f13580a2380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13681000d0 tx=0x7f135c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:39:27.606 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.605+0000 7f1365ffb700 1 -- 172.21.15.134:0/2196555413 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1354095f20 con 0x7f1368100270 2023-08-11T15:39:27.800 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:27.799+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1368108560 con 0x7f135809fec0 2023-08-11T15:39:28.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:28 smithi032 ceph-mon[97103]: osdmap e9: 1 total, 1 up, 1 in 2023-08-11T15:39:28.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:28.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:28.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:28 smithi134 ceph-mon[130447]: osdmap e9: 1 total, 1 up, 1 in 2023-08-11T15:39:28.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:28.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:28.566 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:28.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:28 smithi192 ceph-mon[130806]: osdmap e9: 1 total, 1 up, 1 in 2023-08-11T15:39:28.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:28.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:28.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:29.486 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:29 smithi134 ceph-mon[130447]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:29.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:29 smithi134 ceph-mon[130447]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:29.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:29 smithi032 ceph-mon[97103]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:29.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:29 smithi032 ceph-mon[97103]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:29.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:29 smithi192 ceph-mon[130806]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:29.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:29 smithi192 ceph-mon[130806]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:30.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:30 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/592672693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]: dispatch 2023-08-11T15:39:30.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:30 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]: dispatch 2023-08-11T15:39:30.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:30 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]': finished 2023-08-11T15:39:30.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:30 smithi032 ceph-mon[97103]: osdmap e10: 2 total, 1 up, 2 in 2023-08-11T15:39:30.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:30 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/592672693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]: dispatch 2023-08-11T15:39:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:30 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]: dispatch 2023-08-11T15:39:30.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:30 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]': finished 2023-08-11T15:39:30.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:30 smithi192 ceph-mon[130806]: osdmap e10: 2 total, 1 up, 2 in 2023-08-11T15:39:30.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:30.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:30 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/592672693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]: dispatch 2023-08-11T15:39:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:30 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]: dispatch 2023-08-11T15:39:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:30 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7af16e40-e09e-4ef1-ab51-bd03df34e972"}]': finished 2023-08-11T15:39:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:30 smithi134 ceph-mon[130447]: osdmap e10: 2 total, 1 up, 2 in 2023-08-11T15:39:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:31.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:31 smithi032 ceph-mon[97103]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:31.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:31 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/296303407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:31.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:31 smithi192 ceph-mon[130806]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:31 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/296303407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:31.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:31 smithi134 ceph-mon[130447]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:31.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:31 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/296303407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:33.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:32 smithi032 ceph-mon[97103]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:33.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:32 smithi192 ceph-mon[130806]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:33.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:32 smithi134 ceph-mon[130447]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:35.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:35 smithi192 ceph-mon[130806]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:35.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:35 smithi134 ceph-mon[130447]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:35.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:35 smithi032 ceph-mon[97103]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:37.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:37 smithi192 ceph-mon[130806]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:37.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:37 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-11T15:39:37.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:37 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:37.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:37 smithi134 ceph-mon[130447]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:37.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:37 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-11T15:39:37.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:37 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:37.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:37 smithi032 ceph-mon[97103]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:37.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:37 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-11T15:39:37.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:37 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:38 smithi192 ceph-mon[130806]: Deploying daemon osd.1 on smithi134 2023-08-11T15:39:38.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:38 smithi134 ceph-mon[130447]: Deploying daemon osd.1 on smithi134 2023-08-11T15:39:38.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:38 smithi032 ceph-mon[97103]: Deploying daemon osd.1 on smithi134 2023-08-11T15:39:39.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:39 smithi192 ceph-mon[130806]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:39.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:39 smithi134 ceph-mon[130447]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:39.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:39 smithi032 ceph-mon[97103]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:41.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:41 smithi192 ceph-mon[130806]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:41.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:41 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:41.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:41 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:41.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:41 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:41.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:41 smithi134 ceph-mon[130447]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:41.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:41 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:41 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:41 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:41.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:41 smithi032 ceph-mon[97103]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:41.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:41 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:41.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:41 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:41.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:41 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:42.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:42 smithi134 ceph-mon[130447]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:43.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:42 smithi032 ceph-mon[97103]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:43.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:42 smithi192 ceph-mon[130806]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:43.515 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 1 on host 'smithi134' 2023-08-11T15:39:43.516 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.513+0000 7f1365ffb700 1 -- 172.21.15.134:0/2196555413 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1368108560 con 0x7f135809fec0 2023-08-11T15:39:43.517 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.515+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f135809fec0 msgr2=0x7f13580a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:43.517 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.515+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f135809fec0 0x7f13580a2380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13681000d0 tx=0x7f135c00b410 comp rx=0 tx=0).stop 2023-08-11T15:39:43.517 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.515+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 msgr2=0x7f1368196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:43.517 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.515+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f1368196c10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1354009990 tx=0x7f1354004730 comp rx=0 tx=0).stop 2023-08-11T15:39:43.518 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 shutdown_connections 2023-08-11T15:39:43.518 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f135809fec0 0x7f13580a2380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:43.518 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1368100c10 0x7f136819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:43.518 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13680ff070 0x7f13681966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:43.518 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 --2- 172.21.15.134:0/2196555413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1368100270 0x7f1368196c10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:43.518 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 >> 172.21.15.134:0/2196555413 conn(0x7f13680fa610 msgr2=0x7f1368107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:39:43.519 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 shutdown_connections 2023-08-11T15:39:43.519 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:39:43.516+0000 7f136ebbc700 1 -- 172.21.15.134:0/2196555413 wait complete. 2023-08-11T15:39:43.979 DEBUG:teuthology.orchestra.run.smithi134:osd.1> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.1.service 2023-08-11T15:39:43.982 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2023-08-11T15:39:43.982 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-11T15:39:44.015 INFO:journalctl@ceph.osd.1.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:39:44.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:44.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:44.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:44.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:44.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:44.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:44.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:44.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:44.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:44.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:39:44.541 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:39:45.018 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:44 smithi192 ceph-mon[130806]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:45.018 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:44 smithi192 ceph-mon[130806]: from='osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T15:39:45.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:44 smithi192 ceph-mon[130806]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T15:39:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:44 smithi134 ceph-mon[130447]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:44 smithi134 ceph-mon[130447]: from='osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T15:39:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:44 smithi134 ceph-mon[130447]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T15:39:45.221 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:39:44 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-1[135518]: 2023-08-11T15:39:44.803+0000 7f56cc7f9640 -1 osd.1 0 log_to_monitors true 2023-08-11T15:39:45.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:44 smithi032 ceph-mon[97103]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:45.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:44 smithi032 ceph-mon[97103]: from='osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T15:39:45.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:44 smithi032 ceph-mon[97103]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T15:39:46.029 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-11T15:39:46.030 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-11T15:39:46.030 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-08-11T15:39:46.030 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-08-11T15:39:46.030 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110018 s, 953 MB/s 2023-08-11T15:39:46.030 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-08-11T15:39:46.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:45 smithi134 ceph-mon[130447]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-11T15:39:46.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:45 smithi134 ceph-mon[130447]: osdmap e11: 2 total, 1 up, 2 in 2023-08-11T15:39:46.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:45 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:46.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:45 smithi134 ceph-mon[130447]: from='osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:39:46.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:45 smithi134 ceph-mon[130447]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:39:46.178 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:39:45 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-1[135518]: 2023-08-11T15:39:45.975+0000 7f56c1e71700 -1 osd.1 0 waiting for initial osdmap 2023-08-11T15:39:46.178 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:39:45 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-1[135518]: 2023-08-11T15:39:45.977+0000 7f56bb461700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:39:46.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:45 smithi032 ceph-mon[97103]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-11T15:39:46.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:45 smithi032 ceph-mon[97103]: osdmap e11: 2 total, 1 up, 2 in 2023-08-11T15:39:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:45 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:45 smithi032 ceph-mon[97103]: from='osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:39:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:45 smithi032 ceph-mon[97103]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:39:46.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:45 smithi192 ceph-mon[130806]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-11T15:39:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:45 smithi192 ceph-mon[130806]: osdmap e11: 2 total, 1 up, 2 in 2023-08-11T15:39:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:45 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:45 smithi192 ceph-mon[130806]: from='osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:39:46.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:45 smithi192 ceph-mon[130806]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:39:46.388 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2023-08-11T15:39:46.944 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:39:47.297 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:39:47.297 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: osdmap e12: 2 total, 1 up, 2 in 2023-08-11T15:39:47.297 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:47.297 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:47.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:47.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi134 2023-08-11T15:39:47.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:39:47.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.299 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:47.299 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:47.299 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:39:47.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: osdmap e12: 2 total, 1 up, 2 in 2023-08-11T15:39:47.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:47.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:47.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:47.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi134 2023-08-11T15:39:47.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:39:47.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:47.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:47.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:39:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: osdmap e12: 2 total, 1 up, 2 in 2023-08-11T15:39:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T15:39:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi134 2023-08-11T15:39:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:39:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:39:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:39:47.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 -- 172.21.15.192:0/2229260662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b81066a0 msgr2=0x7fc1b8108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:47.994 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 --2- 172.21.15.192:0/2229260662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b81066a0 0x7fc1b8108b10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000b0a0 tx=0x7fc1b000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:39:47.994 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 -- 172.21.15.192:0/2229260662 shutdown_connections 2023-08-11T15:39:47.994 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 --2- 172.21.15.192:0/2229260662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8101440 0x7fc1b8103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:47.995 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 --2- 172.21.15.192:0/2229260662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b81066a0 0x7fc1b8108b10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:47.995 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 --2- 172.21.15.192:0/2229260662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8103d70 0x7fc1b8106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:47.995 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 -- 172.21.15.192:0/2229260662 >> 172.21.15.192:0/2229260662 conn(0x7fc1b80fae30 msgr2=0x7fc1b80fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:39:47.995 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 -- 172.21.15.192:0/2229260662 shutdown_connections 2023-08-11T15:39:47.996 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.988+0000 7fc1bea76700 1 -- 172.21.15.192:0/2229260662 wait complete. 2023-08-11T15:39:47.996 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 Processor -- start 2023-08-11T15:39:47.996 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 -- start start 2023-08-11T15:39:47.996 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b8101440 0x7fc1b81967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:47.997 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8103d70 0x7fc1b8196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:47.997 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b81066a0 0x7fc1b819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:47.997 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b819e360 con 0x7fc1b81066a0 2023-08-11T15:39:47.998 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b819e4a0 con 0x7fc1b8103d70 2023-08-11T15:39:47.998 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bea76700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b819e630 con 0x7fc1b8101440 2023-08-11T15:39:47.998 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b8101440 0x7fc1b81967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:47.998 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b8101440 0x7fc1b81967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57438/0 (socket says 172.21.15.192:57438) 2023-08-11T15:39:47.998 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 -- 172.21.15.192:0/3222099724 learned_addr learned my addr 172.21.15.192:0/3222099724 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:39:47.998 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1b7fff700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8103d70 0x7fc1b8196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:47.998 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 -- 172.21.15.192:0/3222099724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8103d70 msgr2=0x7fc1b8196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:47.999 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8103d70 0x7fc1b8196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:47.999 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bd013700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b81066a0 0x7fc1b819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:47.999 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 -- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b81066a0 msgr2=0x7fc1b819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:39:47.999 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b81066a0 0x7fc1b819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:39:47.999 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1bc812700 1 -- 172.21.15.192:0/3222099724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0009d00 con 0x7fc1b8101440 2023-08-11T15:39:47.999 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.989+0000 7fc1b7fff700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8103d70 0x7fc1b8196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:39:47.999 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.990+0000 7fc1bc812700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b8101440 0x7fc1b81967d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c950 tx=0x7fc1ac00cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:39:48.000 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.990+0000 7fc1b5ffb700 1 -- 172.21.15.192:0/3222099724 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac007ac0 con 0x7fc1b8101440 2023-08-11T15:39:48.000 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.990+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b819e910 con 0x7fc1b8101440 2023-08-11T15:39:48.000 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.990+0000 7fc1b5ffb700 1 -- 172.21.15.192:0/3222099724 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1ac004cf0 con 0x7fc1b8101440 2023-08-11T15:39:48.000 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.990+0000 7fc1b5ffb700 1 -- 172.21.15.192:0/3222099724 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac005580 con 0x7fc1b8101440 2023-08-11T15:39:48.000 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.990+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b819ee60 con 0x7fc1b8101440 2023-08-11T15:39:48.000 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.991+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b80fc9b0 con 0x7fc1b8101440 2023-08-11T15:39:48.001 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.992+0000 7fc1b5ffb700 1 -- 172.21.15.192:0/3222099724 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7fc1ac007c20 con 0x7fc1b8101440 2023-08-11T15:39:48.002 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.992+0000 7fc1b5ffb700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc1a809fec0 0x7fc1a80a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:39:48.002 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.993+0000 7fc1b5ffb700 1 -- 172.21.15.192:0/3222099724 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fc1ac0c96b0 con 0x7fc1b8101440 2023-08-11T15:39:48.003 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.993+0000 7fc1b7fff700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc1a809fec0 0x7fc1a80a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:39:48.003 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.994+0000 7fc1b7fff700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc1a809fec0 0x7fc1a80a2380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4005fd0 tx=0x7fc1a4005ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:39:48.003 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:47.997+0000 7fc1b5ffb700 1 -- 172.21.15.192:0/3222099724 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc1ac094bf0 con 0x7fc1b8101440 2023-08-11T15:39:48.193 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:39:48.192+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b802cd20 con 0x7fc1a809fec0 2023-08-11T15:39:48.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:47 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:39:48.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:47 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:39:48.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:47 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi134 to 17168M 2023-08-11T15:39:48.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:47 smithi032 ceph-mon[97103]: osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] boot 2023-08-11T15:39:48.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:47 smithi032 ceph-mon[97103]: osdmap e13: 2 total, 2 up, 2 in 2023-08-11T15:39:48.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:47 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:39:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:47 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:39:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:47 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi134 to 17168M 2023-08-11T15:39:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:47 smithi192 ceph-mon[130806]: osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] boot 2023-08-11T15:39:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:47 smithi192 ceph-mon[130806]: osdmap e13: 2 total, 2 up, 2 in 2023-08-11T15:39:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:48.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:47 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:39:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:47 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:39:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:47 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi134 to 17168M 2023-08-11T15:39:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:47 smithi134 ceph-mon[130447]: osd.1 [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] boot 2023-08-11T15:39:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:47 smithi134 ceph-mon[130447]: osdmap e13: 2 total, 2 up, 2 in 2023-08-11T15:39:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:39:49.275 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:48 smithi192 ceph-mon[130806]: osdmap e14: 2 total, 2 up, 2 in 2023-08-11T15:39:49.276 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:48 smithi192 ceph-mon[130806]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:49.276 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:48 smithi192 ceph-mon[130806]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:49.276 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:48 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:49.277 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:48 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:49.277 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:48 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:49.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:48 smithi032 ceph-mon[97103]: osdmap e14: 2 total, 2 up, 2 in 2023-08-11T15:39:49.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:48 smithi032 ceph-mon[97103]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:49.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:48 smithi032 ceph-mon[97103]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:49.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:48 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:49.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:48 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:49.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:48 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:49.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:48 smithi134 ceph-mon[130447]: osdmap e14: 2 total, 2 up, 2 in 2023-08-11T15:39:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:48 smithi134 ceph-mon[130447]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:48 smithi134 ceph-mon[130447]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:39:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:48 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:39:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:48 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:39:49.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:48 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:51.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:51 smithi192 ceph-mon[130806]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:51 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/4099681352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]: dispatch 2023-08-11T15:39:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:51 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]: dispatch 2023-08-11T15:39:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:51 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]': finished 2023-08-11T15:39:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:51 smithi192 ceph-mon[130806]: osdmap e15: 3 total, 2 up, 3 in 2023-08-11T15:39:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:39:51.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:51 smithi134 ceph-mon[130447]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:51 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/4099681352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]: dispatch 2023-08-11T15:39:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:51 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]: dispatch 2023-08-11T15:39:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:51 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]': finished 2023-08-11T15:39:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:51 smithi134 ceph-mon[130447]: osdmap e15: 3 total, 2 up, 3 in 2023-08-11T15:39:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:39:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:51 smithi032 ceph-mon[97103]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:51 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/4099681352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]: dispatch 2023-08-11T15:39:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:51 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]: dispatch 2023-08-11T15:39:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:51 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8861d523-118d-4cd5-b4fa-73fbfced3c76"}]': finished 2023-08-11T15:39:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:51 smithi032 ceph-mon[97103]: osdmap e15: 3 total, 2 up, 3 in 2023-08-11T15:39:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:39:52.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:52 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/891888961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:52.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:52 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/891888961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:52 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/891888961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:39:53.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:53 smithi192 ceph-mon[130806]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:53.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:53 smithi134 ceph-mon[130447]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:53 smithi032 ceph-mon[97103]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:55.329 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:55 smithi192 ceph-mon[130806]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:55.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:55 smithi134 ceph-mon[130447]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:55 smithi032 ceph-mon[97103]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:57.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:57 smithi192 ceph-mon[130806]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:57.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-11T15:39:57.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:57.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:57 smithi134 ceph-mon[130447]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-11T15:39:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:57 smithi032 ceph-mon[97103]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-11T15:39:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:39:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:58 smithi192 ceph-mon[130806]: Deploying daemon osd.2 on smithi192 2023-08-11T15:39:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:58 smithi134 ceph-mon[130447]: Deploying daemon osd.2 on smithi192 2023-08-11T15:39:58.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:58 smithi032 ceph-mon[97103]: Deploying daemon osd.2 on smithi192 2023-08-11T15:39:59.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:39:59 smithi192 ceph-mon[130806]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:59.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:39:59 smithi134 ceph-mon[130447]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:39:59.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:39:59 smithi032 ceph-mon[97103]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:00.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:00 smithi192 ceph-mon[130806]: overall HEALTH_OK 2023-08-11T15:40:00.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:00 smithi134 ceph-mon[130447]: overall HEALTH_OK 2023-08-11T15:40:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:00 smithi032 ceph-mon[97103]: overall HEALTH_OK 2023-08-11T15:40:01.346 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:01 smithi192 ceph-mon[130806]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:01.347 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:01 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:01.347 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:01 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:01.347 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:01 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:01.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:01 smithi134 ceph-mon[130447]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:01.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:01 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:01.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:01 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:01.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:01 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:01 smithi032 ceph-mon[97103]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:01 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:01 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:01 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:03.044 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:02 smithi192 ceph-mon[130806]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:03.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:02 smithi032 ceph-mon[97103]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:03.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:02 smithi134 ceph-mon[130447]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:03.458 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2023-08-11T15:40:03.458 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.456+0000 7fc1b5ffb700 1 -- 172.21.15.192:0/3222099724 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc1b802cd20 con 0x7fc1a809fec0 2023-08-11T15:40:03.460 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc1a809fec0 msgr2=0x7fc1a80a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:03.460 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc1a809fec0 0x7fc1a80a2380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4005fd0 tx=0x7fc1a4005ee0 comp rx=0 tx=0).stop 2023-08-11T15:40:03.461 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b8101440 msgr2=0x7fc1b81967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:03.461 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b8101440 0x7fc1b81967d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c950 tx=0x7fc1ac00cd10 comp rx=0 tx=0).stop 2023-08-11T15:40:03.461 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 shutdown_connections 2023-08-11T15:40:03.462 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc1a809fec0 0x7fc1a80a2380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:03.462 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b81066a0 0x7fc1b819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:03.462 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1b8101440 0x7fc1b81967d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:03.463 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 --2- 172.21.15.192:0/3222099724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8103d70 0x7fc1b8196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:03.463 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 >> 172.21.15.192:0/3222099724 conn(0x7fc1b80fae30 msgr2=0x7fc1b8102130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:40:03.463 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.458+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 shutdown_connections 2023-08-11T15:40:03.464 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:03.459+0000 7fc1bea76700 1 -- 172.21.15.192:0/3222099724 wait complete. 2023-08-11T15:40:03.824 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.2.service 2023-08-11T15:40:03.827 INFO:tasks.cephadm:Deploying osd.3 on smithi032 with /dev/vg_nvme/lv_3... 2023-08-11T15:40:03.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-11T15:40:03.857 INFO:journalctl@ceph.osd.2.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:40:04.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:04.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:04.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:04.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:04.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:04.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:40:04.936 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-2[136609]: 2023-08-11T15:40:04.882+0000 7feef8a75640 -1 osd.2 0 log_to_monitors true 2023-08-11T15:40:05.134 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:04 smithi032 ceph-mon[97103]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:05.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:04 smithi032 ceph-mon[97103]: from='osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T15:40:05.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:04 smithi032 ceph-mon[97103]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T15:40:05.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:04 smithi134 ceph-mon[130447]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:04 smithi134 ceph-mon[130447]: from='osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T15:40:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:04 smithi134 ceph-mon[130447]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T15:40:05.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:04 smithi192 ceph-mon[130806]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:04 smithi192 ceph-mon[130806]: from='osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T15:40:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:04 smithi192 ceph-mon[130806]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T15:40:05.945 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:05 smithi192 ceph-mon[130806]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-11T15:40:05.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:05 smithi192 ceph-mon[130806]: osdmap e16: 3 total, 2 up, 3 in 2023-08-11T15:40:05.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:05.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:05 smithi192 ceph-mon[130806]: from='osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:40:05.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:05 smithi192 ceph-mon[130806]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:40:06.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:05 smithi032 ceph-mon[97103]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-11T15:40:06.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:05 smithi032 ceph-mon[97103]: osdmap e16: 3 total, 2 up, 3 in 2023-08-11T15:40:06.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:06.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:05 smithi032 ceph-mon[97103]: from='osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:40:06.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:05 smithi032 ceph-mon[97103]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:40:06.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:05 smithi134 ceph-mon[130447]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-11T15:40:06.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:05 smithi134 ceph-mon[130447]: osdmap e16: 3 total, 2 up, 3 in 2023-08-11T15:40:06.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:06.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:05 smithi134 ceph-mon[130447]: from='osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:40:06.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:05 smithi134 ceph-mon[130447]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:40:06.333 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-2[136609]: 2023-08-11T15:40:05.943+0000 7feeee0ed700 -1 osd.2 0 waiting for initial osdmap 2023-08-11T15:40:06.333 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-2[136609]: 2023-08-11T15:40:05.947+0000 7feee66db700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:40:06.502 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-11T15:40:06.503 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-11T15:40:06.503 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-08-11T15:40:06.503 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-08-11T15:40:06.503 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0300741 s, 349 MB/s 2023-08-11T15:40:06.503 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-08-11T15:40:07.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi032:vg_nvme/lv_3 2023-08-11T15:40:07.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:40:07.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: osdmap e17: 3 total, 2 up, 3 in 2023-08-11T15:40:07.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:07.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:07.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi192 2023-08-11T15:40:07.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:07.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi192 to 17168M 2023-08-11T15:40:07.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:07.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:07.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:06 smithi134 ceph-mon[130447]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:40:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: osdmap e17: 3 total, 2 up, 3 in 2023-08-11T15:40:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi192 2023-08-11T15:40:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:07.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi192 to 17168M 2023-08-11T15:40:07.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:07.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:07.306 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.306 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:06 smithi032 ceph-mon[97103]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:07.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:40:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: osdmap e17: 3 total, 2 up, 3 in 2023-08-11T15:40:07.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:07.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:07.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi192 2023-08-11T15:40:07.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:07.337 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi192 to 17168M 2023-08-11T15:40:07.337 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.337 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:07.337 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:07.338 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:07.338 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:06 smithi192 ceph-mon[130806]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T15:40:07.783 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:40:08.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:07 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:40:08.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:07 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:40:08.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:07 smithi134 ceph-mon[130447]: osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] boot 2023-08-11T15:40:08.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:07 smithi134 ceph-mon[130447]: osdmap e18: 3 total, 3 up, 3 in 2023-08-11T15:40:08.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:08.283 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:07 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:40:08.283 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:07 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:40:08.284 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:07 smithi032 ceph-mon[97103]: osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] boot 2023-08-11T15:40:08.284 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:07 smithi032 ceph-mon[97103]: osdmap e18: 3 total, 3 up, 3 in 2023-08-11T15:40:08.284 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:07 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:40:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:07 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:40:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:07 smithi192 ceph-mon[130806]: osd.2 [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] boot 2023-08-11T15:40:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:07 smithi192 ceph-mon[130806]: osdmap e18: 3 total, 3 up, 3 in 2023-08-11T15:40:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:40:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/2312823334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4101bb0 msgr2=0x7f3bd4102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/2312823334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4101bb0 0x7f3bd4102010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40099c0 tx=0x7f3bc4009cd0 comp rx=0 tx=0).stop 2023-08-11T15:40:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/2312823334 shutdown_connections 2023-08-11T15:40:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/2312823334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd4102550 0x7f3bd4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/2312823334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd41009b0 0x7f3bd4100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/2312823334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4101bb0 0x7f3bd4102010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/2312823334 >> 172.21.15.32:0/2312823334 conn(0x7f3bd40fbf10 msgr2=0x7f3bd40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:40:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/2312823334 shutdown_connections 2023-08-11T15:40:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.215+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/2312823334 wait complete. 2023-08-11T15:40:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.216+0000 7f3bdb8f9700 1 Processor -- start 2023-08-11T15:40:09.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bdb8f9700 1 -- start start 2023-08-11T15:40:09.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bdb8f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd41009b0 0x7f3bd4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bdb8f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd4101bb0 0x7f3bd4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bdb8f9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd4102550 0x7f3bd419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bdb8f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41a03d0 con 0x7f3bd41009b0 2023-08-11T15:40:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bdb8f9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41a0510 con 0x7f3bd4102550 2023-08-11T15:40:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bdb8f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41a06a0 con 0x7f3bd4101bb0 2023-08-11T15:40:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bd9695700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd41009b0 0x7f3bd4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bd9695700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd41009b0 0x7f3bd4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36502/0 (socket says 172.21.15.32:36502) 2023-08-11T15:40:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bd9695700 1 -- 172.21.15.32:0/4056137999 learned_addr learned my addr 172.21.15.32:0/4056137999 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:40:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.217+0000 7f3bd8e94700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd4101bb0 0x7f3bd4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd9e96700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd4102550 0x7f3bd419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd9695700 1 -- 172.21.15.32:0/4056137999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd4101bb0 msgr2=0x7f3bd4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd9695700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd4101bb0 0x7f3bd4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd9695700 1 -- 172.21.15.32:0/4056137999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd4102550 msgr2=0x7f3bd419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd9695700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd4102550 0x7f3bd419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd9695700 1 -- 172.21.15.32:0/4056137999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40095f0 con 0x7f3bd41009b0 2023-08-11T15:40:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd8e94700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd4101bb0 0x7f3bd4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:40:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bd9695700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd41009b0 0x7f3bd4198840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e9a0 tx=0x7f3bd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:40:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0009790 con 0x7f3bd41009b0 2023-08-11T15:40:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd41a0980 con 0x7f3bd41009b0 2023-08-11T15:40:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bd00044e0 con 0x7f3bd41009b0 2023-08-11T15:40:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0003d20 con 0x7f3bd41009b0 2023-08-11T15:40:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.218+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd41a0ed0 con 0x7f3bd41009b0 2023-08-11T15:40:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.219+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd4192fa0 con 0x7f3bd41009b0 2023-08-11T15:40:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.223+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 15) v1 ==== 134615+0+0 (secure 0 0 0) 0x7f3bd0003e80 con 0x7f3bd41009b0 2023-08-11T15:40:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.224+0000 7f3bca7fc700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3bc009fec0 0x7f3bc00a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.225+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f3bd0014070 con 0x7f3bd41009b0 2023-08-11T15:40:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.225+0000 7f3bd8e94700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3bc009fec0 0x7f3bc00a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.225+0000 7f3bd8e94700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3bc009fec0 0x7f3bc00a2380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b580 tx=0x7f3bc4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:40:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.227+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3bd00952e0 con 0x7f3bd41009b0 2023-08-11T15:40:09.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:09 smithi032 ceph-mon[97103]: osdmap e19: 3 total, 3 up, 3 in 2023-08-11T15:40:09.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:09 smithi032 ceph-mon[97103]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:09.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:09 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:40:09.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:09 smithi192 ceph-mon[130806]: osdmap e19: 3 total, 3 up, 3 in 2023-08-11T15:40:09.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:09 smithi192 ceph-mon[130806]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:09.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:09 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:40:09.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:09.446+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bd402cd70 con 0x7f3bc009fec0 2023-08-11T15:40:09.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:09 smithi134 ceph-mon[130447]: osdmap e19: 3 total, 3 up, 3 in 2023-08-11T15:40:09.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:09 smithi134 ceph-mon[130447]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:09.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:09 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:40:10.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-11T15:40:10.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:10 smithi032 ceph-mon[97103]: osdmap e20: 3 total, 3 up, 3 in 2023-08-11T15:40:10.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:40:10.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:10.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:10.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:10.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-11T15:40:10.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:10 smithi192 ceph-mon[130806]: osdmap e20: 3 total, 3 up, 3 in 2023-08-11T15:40:10.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:40:10.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:10.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:10.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:10.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-11T15:40:10.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:10 smithi134 ceph-mon[130447]: osdmap e20: 3 total, 3 up, 3 in 2023-08-11T15:40:10.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:40:10.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:10.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:10.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:10.791 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:10 smithi032 sudo[112462]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T15:40:10.792 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:10 smithi032 sudo[112462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:10.792 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:10 smithi032 sudo[112462]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:10.792 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:10 smithi032 sudo[112462]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:11 smithi032 ceph-mon[97103]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:11 smithi032 ceph-mon[97103]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:11 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-11T15:40:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:11 smithi032 ceph-mon[97103]: osdmap e21: 3 total, 3 up, 3 in 2023-08-11T15:40:11.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:11 smithi192 ceph-mon[130806]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:11.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:11 smithi192 ceph-mon[130806]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:11.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:11 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-11T15:40:11.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:11 smithi192 ceph-mon[130806]: osdmap e21: 3 total, 3 up, 3 in 2023-08-11T15:40:11.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:11 smithi134 ceph-mon[130447]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:11.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:11 smithi134 ceph-mon[130447]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:11.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:11 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-11T15:40:11.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:11 smithi134 ceph-mon[130447]: osdmap e21: 3 total, 3 up, 3 in 2023-08-11T15:40:11.540 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:11 smithi032 sudo[112495]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T15:40:11.540 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:11 smithi032 sudo[112495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:11.540 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:11 smithi032 sudo[112495]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:11.541 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:40:11 smithi032 sudo[112495]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:11.766 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137697]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T15:40:11.766 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137697]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:11.767 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137697]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:11.767 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137697]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:12.029 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137700]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T15:40:12.029 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:12.029 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137700]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:12.030 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:40:11 smithi134 sudo[137700]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:12.291 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:12 smithi192 ceph-mon[130806]: osdmap e22: 3 total, 3 up, 3 in 2023-08-11T15:40:12.291 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:12.291 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:12.292 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138724]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T15:40:12.292 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:12.292 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138724]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:12.292 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138724]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:12.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:12 smithi032 ceph-mon[97103]: osdmap e22: 3 total, 3 up, 3 in 2023-08-11T15:40:12.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:12.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:12.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:12 smithi134 ceph-mon[130447]: osdmap e22: 3 total, 3 up, 3 in 2023-08-11T15:40:12.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:12.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:12.583 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138727]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T15:40:12.583 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138727]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:12.584 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138727]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:12.584 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:40:12 smithi192 sudo[138727]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:12 smithi032 sudo[112582]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T15:40:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:12 smithi032 sudo[112582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:12 smithi032 sudo[112582]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:12 smithi032 sudo[112582]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:13.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:13.411+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f3bd00956b0 con 0x7f3bd41009b0 2023-08-11T15:40:13.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:13 smithi134 ceph-mon[130447]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:13.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:13.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:13 smithi134 ceph-mon[130447]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:13.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:13 smithi032 sudo[112585]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T15:40:13.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:13 smithi032 sudo[112585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:13.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:13 smithi032 sudo[112585]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:13.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:13 smithi032 sudo[112585]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:13.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:13 smithi032 ceph-mon[97103]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:13.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:13.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:13 smithi032 ceph-mon[97103]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:13.892 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138730]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T15:40:13.893 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138730]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:13.893 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138730]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:13.893 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138730]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:14.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138733]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T15:40:14.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:14.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138733]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:14.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 sudo[138733]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:14.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 ceph-mon[130806]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:14.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:14.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:13 smithi192 ceph-mon[130806]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:14.388 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137703]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T15:40:14.389 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137703]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:14.389 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137703]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:14.389 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137703]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137706]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T15:40:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137706]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T15:40:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137706]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T15:40:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 sudo[137706]: pam_unix(sudo:session): session closed for user root 2023-08-11T15:40:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/43401267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]: dispatch 2023-08-11T15:40:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]: dispatch 2023-08-11T15:40:14.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:14.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]': finished 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: osdmap e23: 4 total, 3 up, 4 in 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:14.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:14.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3644269785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:40:14.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:14 smithi134 ceph-mon[130447]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/43401267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]: dispatch 2023-08-11T15:40:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]: dispatch 2023-08-11T15:40:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]': finished 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: osdmap e23: 4 total, 3 up, 4 in 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:14.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3644269785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:40:14.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:14 smithi032 ceph-mon[97103]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/43401267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]: dispatch 2023-08-11T15:40:14.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]: dispatch 2023-08-11T15:40:14.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d001e237-aefd-4118-b7e4-978433e52622"}]': finished 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: osdmap e23: 4 total, 3 up, 4 in 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:14.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:14.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T15:40:14.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:40:14.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:40:14.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:40:14.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:14.837 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3644269785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:40:14.837 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:14 smithi192 ceph-mon[130806]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T15:40:17.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:17 smithi192 ceph-mon[130806]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:17.396 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:17 smithi032 ceph-mon[97103]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:17.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:17 smithi134 ceph-mon[130447]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:19.224 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:19 smithi032 ceph-mon[97103]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 57 KiB/s wr, 5 op/s 2023-08-11T15:40:19.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:19 smithi192 ceph-mon[130806]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 57 KiB/s wr, 5 op/s 2023-08-11T15:40:19.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:19 smithi134 ceph-mon[130447]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 57 KiB/s wr, 5 op/s 2023-08-11T15:40:21.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:21 smithi032 ceph-mon[97103]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 40 KiB/s wr, 3 op/s 2023-08-11T15:40:21.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:21 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-11T15:40:21.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:21 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:21.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:21 smithi192 ceph-mon[130806]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 40 KiB/s wr, 3 op/s 2023-08-11T15:40:21.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:21 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-11T15:40:21.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:21 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:21 smithi134 ceph-mon[130447]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 40 KiB/s wr, 3 op/s 2023-08-11T15:40:21.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:21 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-11T15:40:21.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:21 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:22.281 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:22 smithi032 ceph-mon[97103]: Deploying daemon osd.3 on smithi032 2023-08-11T15:40:22.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:22 smithi192 ceph-mon[130806]: Deploying daemon osd.3 on smithi032 2023-08-11T15:40:22.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:22 smithi134 ceph-mon[130447]: Deploying daemon osd.3 on smithi032 2023-08-11T15:40:23.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:23 smithi032 ceph-mon[97103]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 30 KiB/s wr, 2 op/s 2023-08-11T15:40:23.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:23 smithi192 ceph-mon[130806]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 30 KiB/s wr, 2 op/s 2023-08-11T15:40:23.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:23 smithi134 ceph-mon[130447]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 30 KiB/s wr, 2 op/s 2023-08-11T15:40:25.438 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:25 smithi032 ceph-mon[97103]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 964 B/s rd, 25 KiB/s wr, 2 op/s 2023-08-11T15:40:25.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:25 smithi134 ceph-mon[130447]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 964 B/s rd, 25 KiB/s wr, 2 op/s 2023-08-11T15:40:25.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:25 smithi192 ceph-mon[130806]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 964 B/s rd, 25 KiB/s wr, 2 op/s 2023-08-11T15:40:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:27 smithi032 ceph-mon[97103]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-11T15:40:27.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:27 smithi134 ceph-mon[130447]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-11T15:40:27.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:27 smithi192 ceph-mon[130806]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-11T15:40:28.716 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:28.717 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:28.717 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:28.717 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:28 smithi032 ceph-mon[97103]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-11T15:40:28.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:28.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:28.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:28.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:28 smithi134 ceph-mon[130447]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-11T15:40:28.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:28.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:28.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:28.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:28 smithi192 ceph-mon[130806]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-11T15:40:30.922 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 3 on host 'smithi032' 2023-08-11T15:40:30.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.920+0000 7f3bca7fc700 1 -- 172.21.15.32:0/4056137999 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3bd402cd70 con 0x7f3bc009fec0 2023-08-11T15:40:30.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3bc009fec0 msgr2=0x7f3bc00a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:30.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3bc009fec0 0x7f3bc00a2380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b580 tx=0x7f3bc4005fb0 comp rx=0 tx=0).stop 2023-08-11T15:40:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd41009b0 msgr2=0x7f3bd4198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd41009b0 0x7f3bd4198840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e9a0 tx=0x7f3bd000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:40:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 shutdown_connections 2023-08-11T15:40:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3bc009fec0 0x7f3bc00a2380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd41009b0 0x7f3bd4198840 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:30.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd4102550 0x7f3bd419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 --2- 172.21.15.32:0/4056137999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd4101bb0 0x7f3bd4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 >> 172.21.15.32:0/4056137999 conn(0x7f3bd40fbf10 msgr2=0x7f3bd4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:40:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 shutdown_connections 2023-08-11T15:40:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:40:30.923+0000 7f3bdb8f9700 1 -- 172.21.15.32:0/4056137999 wait complete. 2023-08-11T15:40:31.396 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:31.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:31.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:31.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.398 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.398 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:31.427 DEBUG:teuthology.orchestra.run.smithi032:osd.3> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.3.service 2023-08-11T15:40:31.434 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_3... 2023-08-11T15:40:31.434 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-11T15:40:31.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:31.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:31.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:31.467 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:31.516 INFO:journalctl@ceph.osd.3.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:40:31.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:31.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:31.999 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:40:32.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:32 smithi032 ceph-mon[97103]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T15:40:32.393 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:32 smithi134 ceph-mon[130447]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T15:40:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:32 smithi192 ceph-mon[130806]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T15:40:33.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:33 smithi134 ceph-mon[130447]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:33.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:33 smithi134 ceph-mon[130447]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-11T15:40:33.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:33 smithi134 ceph-mon[130447]: osdmap e24: 4 total, 3 up, 4 in 2023-08-11T15:40:33.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:33 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:33.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:33 smithi134 ceph-mon[130447]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:40:33.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:33 smithi032 ceph-mon[97103]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:33.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:33 smithi032 ceph-mon[97103]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-11T15:40:33.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:33 smithi032 ceph-mon[97103]: osdmap e24: 4 total, 3 up, 4 in 2023-08-11T15:40:33.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:33 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:33.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:33 smithi032 ceph-mon[97103]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:40:33.554 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 11 15:40:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-3[114810]: 2023-08-11T15:40:33.149+0000 7f97d9d5d700 -1 osd.3 0 waiting for initial osdmap 2023-08-11T15:40:33.555 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 11 15:40:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-3[114810]: 2023-08-11T15:40:33.155+0000 7f97d234b700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:40:33.560 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-11T15:40:33.560 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-11T15:40:33.560 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-08-11T15:40:33.560 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-08-11T15:40:33.560 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.010889 s, 963 MB/s 2023-08-11T15:40:33.561 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-08-11T15:40:33.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:33 smithi192 ceph-mon[130806]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:33.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:33 smithi192 ceph-mon[130806]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-11T15:40:33.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:33 smithi192 ceph-mon[130806]: osdmap e24: 4 total, 3 up, 4 in 2023-08-11T15:40:33.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:33 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:33.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:33 smithi192 ceph-mon[130806]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:40:34.041 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2023-08-11T15:40:34.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:34 smithi134 ceph-mon[130447]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:40:34.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:34 smithi134 ceph-mon[130447]: osdmap e25: 4 total, 3 up, 4 in 2023-08-11T15:40:34.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:34 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:34.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:34 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:34.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:34 smithi032 ceph-mon[97103]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:40:34.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:34 smithi032 ceph-mon[97103]: osdmap e25: 4 total, 3 up, 4 in 2023-08-11T15:40:34.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:34 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:34.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:34 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:34.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:34 smithi192 ceph-mon[130806]: from='osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:40:34.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:34 smithi192 ceph-mon[130806]: osdmap e25: 4 total, 3 up, 4 in 2023-08-11T15:40:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:34.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:34 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:40:35.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:40:35.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:40:35.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] boot 2023-08-11T15:40:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: osdmap e26: 4 total, 4 up, 4 in 2023-08-11T15:40:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi032 2023-08-11T15:40:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:35.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:35.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi032 to 8636M 2023-08-11T15:40:35.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:35.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:35.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:35 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:40:35.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:40:35.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:35.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] boot 2023-08-11T15:40:35.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: osdmap e26: 4 total, 4 up, 4 in 2023-08-11T15:40:35.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:35.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi032 2023-08-11T15:40:35.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:35.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:35.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi032 to 8636M 2023-08-11T15:40:35.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:35.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:35.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:35 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:40:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:40:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T15:40:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: osd.3 [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] boot 2023-08-11T15:40:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: osdmap e26: 4 total, 4 up, 4 in 2023-08-11T15:40:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:40:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi032 2023-08-11T15:40:35.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:35.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:35.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi032 to 8636M 2023-08-11T15:40:35.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:35.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:35.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:35 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:35.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 -- 172.21.15.134:0/2510752466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9500100ec0 msgr2=0x7f95001012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:35.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 --2- 172.21.15.134:0/2510752466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9500100ec0 0x7f95001012e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009990 tx=0x7f94f4009ca0 comp rx=0 tx=0).stop 2023-08-11T15:40:35.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 -- 172.21.15.134:0/2510752466 shutdown_connections 2023-08-11T15:40:35.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 --2- 172.21.15.134:0/2510752466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95001004e0 0x7f9500107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:35.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 --2- 172.21.15.134:0/2510752466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9500100ec0 0x7f95001012e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:35.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 --2- 172.21.15.134:0/2510752466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95000ffb20 0x7f95000fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:35.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 -- 172.21.15.134:0/2510752466 >> 172.21.15.134:0/2510752466 conn(0x7f95000fa5d0 msgr2=0x7f95000fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:40:35.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 -- 172.21.15.134:0/2510752466 shutdown_connections 2023-08-11T15:40:35.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.879+0000 7f9505115700 1 -- 172.21.15.134:0/2510752466 wait complete. 2023-08-11T15:40:35.884 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 Processor -- start 2023-08-11T15:40:35.884 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 -- start start 2023-08-11T15:40:35.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95000ffb20 0x7f9500195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:35.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95001004e0 0x7f9500195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:35.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9500100ec0 0x7f950019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:35.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950019d1a0 con 0x7f95001004e0 2023-08-11T15:40:35.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950019d2e0 con 0x7f95000ffb20 2023-08-11T15:40:35.886 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f9505115700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950019d470 con 0x7f9500100ec0 2023-08-11T15:40:35.886 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f94fed9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95000ffb20 0x7f9500195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:35.886 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f94fed9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95000ffb20 0x7f9500195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53316/0 (socket says 172.21.15.134:53316) 2023-08-11T15:40:35.886 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.880+0000 7f94fed9d700 1 -- 172.21.15.134:0/1577685269 learned_addr learned my addr 172.21.15.134:0/1577685269 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-08-11T15:40:35.886 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94ff59e700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9500100ec0 0x7f950019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:35.886 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fed9d700 1 -- 172.21.15.134:0/1577685269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9500100ec0 msgr2=0x7f950019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:35.886 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fe59c700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95001004e0 0x7f9500195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fed9d700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9500100ec0 0x7f950019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fed9d700 1 -- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95001004e0 msgr2=0x7f9500195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fed9d700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95001004e0 0x7f9500195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fed9d700 1 -- 172.21.15.134:0/1577685269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f40095f0 con 0x7f95000ffb20 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fe59c700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95001004e0 0x7f9500195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94fed9d700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95000ffb20 0x7f9500195610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94f4000c00 tx=0x7f94f400f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94e7fff700 1 -- 172.21.15.134:0/1577685269 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f401d070 con 0x7f95000ffb20 2023-08-11T15:40:35.887 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950019d6f0 con 0x7f95000ffb20 2023-08-11T15:40:35.888 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94e7fff700 1 -- 172.21.15.134:0/1577685269 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f400fca0 con 0x7f95000ffb20 2023-08-11T15:40:35.888 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f94e7fff700 1 -- 172.21.15.134:0/1577685269 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4017720 con 0x7f95000ffb20 2023-08-11T15:40:35.888 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.881+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950019db00 con 0x7f95000ffb20 2023-08-11T15:40:35.888 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.882+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950010b120 con 0x7f95000ffb20 2023-08-11T15:40:35.888 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.883+0000 7f94e7fff700 1 -- 172.21.15.134:0/1577685269 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f94f4017880 con 0x7f95000ffb20 2023-08-11T15:40:35.889 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.883+0000 7f94e7fff700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f94e809ff50 0x7f94e80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:35.889 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.884+0000 7f94e7fff700 1 -- 172.21.15.134:0/1577685269 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f94f40cb550 con 0x7f95000ffb20 2023-08-11T15:40:35.890 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.884+0000 7f94fe59c700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f94e809ff50 0x7f94e80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:35.890 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.884+0000 7f94fe59c700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f94e809ff50 0x7f94e80a2410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007900 tx=0x7f94f0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:40:35.890 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:35.888+0000 7f94e7fff700 1 -- 172.21.15.134:0/1577685269 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f94f405cc30 con 0x7f95000ffb20 2023-08-11T15:40:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:36.084+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9500060d60 con 0x7f94e809ff50 2023-08-11T15:40:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:36 smithi134 ceph-mon[130447]: osdmap e27: 4 total, 4 up, 4 in 2023-08-11T15:40:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:36.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:36.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:36 smithi032 ceph-mon[97103]: osdmap e27: 4 total, 4 up, 4 in 2023-08-11T15:40:36.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:36.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:36.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:36.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:36 smithi192 ceph-mon[130806]: osdmap e27: 4 total, 4 up, 4 in 2023-08-11T15:40:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:37.432 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:37 smithi134 ceph-mon[130447]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:37.433 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:37 smithi134 ceph-mon[130447]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:37.433 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:37 smithi134 ceph-mon[130447]: osdmap e28: 4 total, 4 up, 4 in 2023-08-11T15:40:37.433 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:37 smithi134 ceph-mon[130447]: osdmap e29: 4 total, 4 up, 4 in 2023-08-11T15:40:37.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:37 smithi032 ceph-mon[97103]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:37.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:37 smithi032 ceph-mon[97103]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:37.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:37 smithi032 ceph-mon[97103]: osdmap e28: 4 total, 4 up, 4 in 2023-08-11T15:40:37.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:37 smithi032 ceph-mon[97103]: osdmap e29: 4 total, 4 up, 4 in 2023-08-11T15:40:37.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:37 smithi192 ceph-mon[130806]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:37.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:37 smithi192 ceph-mon[130806]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:37 smithi192 ceph-mon[130806]: osdmap e28: 4 total, 4 up, 4 in 2023-08-11T15:40:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:37 smithi192 ceph-mon[130806]: osdmap e29: 4 total, 4 up, 4 in 2023-08-11T15:40:39.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:38 smithi032 ceph-mon[97103]: osdmap e30: 4 total, 4 up, 4 in 2023-08-11T15:40:39.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:38 smithi032 ceph-mon[97103]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 149 KiB/s, 0 objects/s recovering 2023-08-11T15:40:39.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:38 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/2756092820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]: dispatch 2023-08-11T15:40:39.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:38 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]: dispatch 2023-08-11T15:40:39.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:38 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]': finished 2023-08-11T15:40:39.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:38 smithi032 ceph-mon[97103]: osdmap e31: 5 total, 4 up, 5 in 2023-08-11T15:40:39.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:38 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:39.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:38 smithi192 ceph-mon[130806]: osdmap e30: 4 total, 4 up, 4 in 2023-08-11T15:40:39.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:38 smithi192 ceph-mon[130806]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 149 KiB/s, 0 objects/s recovering 2023-08-11T15:40:39.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:38 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/2756092820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]: dispatch 2023-08-11T15:40:39.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:38 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]: dispatch 2023-08-11T15:40:39.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:38 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]': finished 2023-08-11T15:40:39.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:38 smithi192 ceph-mon[130806]: osdmap e31: 5 total, 4 up, 5 in 2023-08-11T15:40:39.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:38 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:39.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:38 smithi134 ceph-mon[130447]: osdmap e30: 4 total, 4 up, 4 in 2023-08-11T15:40:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:38 smithi134 ceph-mon[130447]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 149 KiB/s, 0 objects/s recovering 2023-08-11T15:40:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:38 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/2756092820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]: dispatch 2023-08-11T15:40:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:38 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]: dispatch 2023-08-11T15:40:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:38 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fc91951-f4cb-4edd-8713-d70f306ba0f5"}]': finished 2023-08-11T15:40:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:38 smithi134 ceph-mon[130447]: osdmap e31: 5 total, 4 up, 5 in 2023-08-11T15:40:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:38 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:40.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:39 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/853305651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:40:40.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:39 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/853305651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:40:40.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:39 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/853305651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:40:41.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:40 smithi032 ceph-mon[97103]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-08-11T15:40:41.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:40 smithi192 ceph-mon[130806]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-08-11T15:40:41.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:40 smithi134 ceph-mon[130447]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-08-11T15:40:42.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:42 smithi134 ceph-mon[130447]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-08-11T15:40:43.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:42 smithi032 ceph-mon[97103]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-08-11T15:40:43.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:42 smithi192 ceph-mon[130806]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-08-11T15:40:45.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:45 smithi192 ceph-mon[130806]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-08-11T15:40:45.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:45 smithi134 ceph-mon[130447]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-08-11T15:40:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:45 smithi032 ceph-mon[97103]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-08-11T15:40:46.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-11T15:40:46.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:46 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:46 smithi192 ceph-mon[130806]: Deploying daemon osd.4 on smithi134 2023-08-11T15:40:46.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-11T15:40:46.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:46 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:46.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:46 smithi134 ceph-mon[130447]: Deploying daemon osd.4 on smithi134 2023-08-11T15:40:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-11T15:40:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:46 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:46.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:46 smithi032 ceph-mon[97103]: Deploying daemon osd.4 on smithi134 2023-08-11T15:40:47.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:47 smithi134 ceph-mon[130447]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:40:47.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:47 smithi032 ceph-mon[97103]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:40:47.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:47 smithi192 ceph-mon[130806]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:40:49.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:49 smithi134 ceph-mon[130447]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:49.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:49 smithi032 ceph-mon[97103]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:49.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:49 smithi192 ceph-mon[130806]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:50.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:50.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:50.638 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:50.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:50 smithi134 ceph-mon[130447]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:50.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:50.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:50.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:50.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:50 smithi032 ceph-mon[97103]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:50.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:50.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:50.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:50.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:50 smithi192 ceph-mon[130806]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:52.200 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.199+0000 7f94e7fff700 1 -- 172.21.15.134:0/1577685269 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9500060d60 con 0x7f94e809ff50 2023-08-11T15:40:52.201 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2023-08-11T15:40:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f94e809ff50 msgr2=0x7f94e80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f94e809ff50 0x7f94e80a2410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007900 tx=0x7f94f0008040 comp rx=0 tx=0).stop 2023-08-11T15:40:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95000ffb20 msgr2=0x7f9500195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95000ffb20 0x7f9500195610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94f4000c00 tx=0x7f94f400f7b0 comp rx=0 tx=0).stop 2023-08-11T15:40:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 shutdown_connections 2023-08-11T15:40:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f94e809ff50 0x7f94e80a2410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9500100ec0 0x7f950019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95000ffb20 0x7f9500195610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 --2- 172.21.15.134:0/1577685269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95001004e0 0x7f9500195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:52.205 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 >> 172.21.15.134:0/1577685269 conn(0x7f95000fa5d0 msgr2=0x7f95000fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:40:52.205 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 shutdown_connections 2023-08-11T15:40:52.205 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:40:52.201+0000 7f9505115700 1 -- 172.21.15.134:0/1577685269 wait complete. 2023-08-11T15:40:52.628 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.4.service 2023-08-11T15:40:52.630 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2023-08-11T15:40:52.631 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-11T15:40:52.670 INFO:journalctl@ceph.osd.4.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:40:52.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:52.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:52.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:52.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:52.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:52.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:52.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:52.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:52.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:53.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:53.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:53.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:53.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:53.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:53.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:53.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:53.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:53.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:40:53.194 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:40:53.426 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:40:53 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-4[140915]: 2023-08-11T15:40:53.361+0000 7fc7165ba640 -1 osd.4 0 log_to_monitors true 2023-08-11T15:40:54.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:53 smithi192 ceph-mon[130806]: from='osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T15:40:54.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:53 smithi192 ceph-mon[130806]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T15:40:54.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:53 smithi134 ceph-mon[130447]: from='osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T15:40:54.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:53 smithi134 ceph-mon[130447]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T15:40:54.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:53 smithi032 ceph-mon[97103]: from='osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T15:40:54.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:53 smithi032 ceph-mon[97103]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T15:40:54.685 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-11T15:40:54.685 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-11T15:40:54.685 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-08-11T15:40:54.685 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-08-11T15:40:54.685 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110077 s, 953 MB/s 2023-08-11T15:40:54.686 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-08-11T15:40:54.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:54 smithi192 ceph-mon[130806]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-11T15:40:54.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:54 smithi192 ceph-mon[130806]: osdmap e32: 5 total, 4 up, 5 in 2023-08-11T15:40:54.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:54.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:54 smithi192 ceph-mon[130806]: from='osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:40:54.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:54 smithi192 ceph-mon[130806]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:40:54.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:54 smithi192 ceph-mon[130806]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:55.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2023-08-11T15:40:55.221 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-4[140915]: 2023-08-11T15:40:54.828+0000 7fc70bc32700 -1 osd.4 0 waiting for initial osdmap 2023-08-11T15:40:55.222 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-4[140915]: 2023-08-11T15:40:54.834+0000 7fc706224700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:40:55.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-mon[130447]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-11T15:40:55.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-mon[130447]: osdmap e32: 5 total, 4 up, 5 in 2023-08-11T15:40:55.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:55.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-mon[130447]: from='osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:40:55.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-mon[130447]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:40:55.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:54 smithi134 ceph-mon[130447]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:55.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:54 smithi032 ceph-mon[97103]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-11T15:40:55.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:54 smithi032 ceph-mon[97103]: osdmap e32: 5 total, 4 up, 5 in 2023-08-11T15:40:55.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:55.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:54 smithi032 ceph-mon[97103]: from='osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:40:55.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:54 smithi032 ceph-mon[97103]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:40:55.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:54 smithi032 ceph-mon[97103]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:55.593 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:40:56.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:40:56.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: osdmap e33: 5 total, 4 up, 5 in 2023-08-11T15:40:56.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:56.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:56.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi134 2023-08-11T15:40:56.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:56.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:56.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi134 to 8584M 2023-08-11T15:40:56.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:56.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:56.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:40:56.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: osdmap e33: 5 total, 4 up, 5 in 2023-08-11T15:40:56.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:56.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:56.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi134 2023-08-11T15:40:56.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:56.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:56.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi134 to 8584M 2023-08-11T15:40:56.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:56.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:56.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:40:56.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: osdmap e33: 5 total, 4 up, 5 in 2023-08-11T15:40:56.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi134 2023-08-11T15:40:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:40:56.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi134 to 8584M 2023-08-11T15:40:56.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:56.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:40:56.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:40:56.791 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 -- 172.21.15.192:0/865803172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f80ff8b0 msgr2=0x7f28f80ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:56.791 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/865803172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f80ff8b0 0x7f28f80ffcd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009990 tx=0x7f28e8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:40:56.791 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 -- 172.21.15.192:0/865803172 shutdown_connections 2023-08-11T15:40:56.792 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/865803172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28f8101450 0x7f28f8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:56.792 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/865803172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f80ff8b0 0x7f28f80ffcd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:56.792 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/865803172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f8100ab0 0x7f28f8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:56.792 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 -- 172.21.15.192:0/865803172 >> 172.21.15.192:0/865803172 conn(0x7f28f80fae30 msgr2=0x7f28f80fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:40:56.792 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 -- 172.21.15.192:0/865803172 shutdown_connections 2023-08-11T15:40:56.792 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.789+0000 7f28fcf0d700 1 -- 172.21.15.192:0/865803172 wait complete. 2023-08-11T15:40:56.793 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 Processor -- start 2023-08-11T15:40:56.793 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 -- start start 2023-08-11T15:40:56.794 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f80ff8b0 0x7f28f8103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:56.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f8100ab0 0x7f28f8101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:56.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28f8101450 0x7f28f81023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:56.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8102b50 con 0x7f28f80ff8b0 2023-08-11T15:40:56.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8102cc0 con 0x7f28f8101450 2023-08-11T15:40:56.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28fcf0d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f81a0710 con 0x7f28f8100ab0 2023-08-11T15:40:56.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f8100ab0 0x7f28f8101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:56.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f8100ab0 0x7f28f8101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57522/0 (socket says 172.21.15.192:57522) 2023-08-11T15:40:56.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 -- 172.21.15.192:0/3156212137 learned_addr learned my addr 172.21.15.192:0/3156212137 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:40:56.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f6d9d700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28f8101450 0x7f28f81023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:56.797 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f659c700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f80ff8b0 0x7f28f8103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:56.797 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 -- 172.21.15.192:0/3156212137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28f8101450 msgr2=0x7f28f81023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:56.798 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28f8101450 0x7f28f81023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:56.798 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 -- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f80ff8b0 msgr2=0x7f28f8103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:40:56.798 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f80ff8b0 0x7f28f8103840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:40:56.798 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 -- 172.21.15.192:0/3156212137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80095f0 con 0x7f28f8100ab0 2023-08-11T15:40:56.798 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f659c700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f80ff8b0 0x7f28f8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:40:56.798 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28f5d9b700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f8100ab0 0x7f28f8101e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00b530 tx=0x7f28ec00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:40:56.798 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.790+0000 7f28e77fe700 1 -- 172.21.15.192:0/3156212137 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec011830 con 0x7f28f8100ab0 2023-08-11T15:40:56.799 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.791+0000 7f28e77fe700 1 -- 172.21.15.192:0/3156212137 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28ec011e50 con 0x7f28f8100ab0 2023-08-11T15:40:56.799 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.791+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f81a09f0 con 0x7f28f8100ab0 2023-08-11T15:40:56.799 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.791+0000 7f28e77fe700 1 -- 172.21.15.192:0/3156212137 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00f2e0 con 0x7f28f8100ab0 2023-08-11T15:40:56.799 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.791+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f81a0fc0 con 0x7f28f8100ab0 2023-08-11T15:40:56.799 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.791+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f810b130 con 0x7f28f8100ab0 2023-08-11T15:40:56.799 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.793+0000 7f28e77fe700 1 -- 172.21.15.192:0/3156212137 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f28ec010380 con 0x7f28f8100ab0 2023-08-11T15:40:56.799 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.793+0000 7f28e77fe700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f28e009ffd0 0x7f28e00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:40:56.800 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.793+0000 7f28e77fe700 1 -- 172.21.15.192:0/3156212137 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f28ec0ca250 con 0x7f28f8100ab0 2023-08-11T15:40:56.800 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.794+0000 7f28f659c700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f28e009ffd0 0x7f28e00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:40:56.801 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.794+0000 7f28f659c700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f28e009ffd0 0x7f28e00a2490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f28e80036f0 tx=0x7f28e8003680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:40:56.801 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.797+0000 7f28e77fe700 1 -- 172.21.15.192:0/3156212137 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f28ec095050 con 0x7f28f8100ab0 2023-08-11T15:40:56.997 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:40:56.994+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f28f8069440 con 0x7f28e009ffd0 2023-08-11T15:40:57.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:56 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:40:57.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:56 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:40:57.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:56 smithi192 ceph-mon[130806]: osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] boot 2023-08-11T15:40:57.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:56 smithi192 ceph-mon[130806]: osdmap e34: 5 total, 5 up, 5 in 2023-08-11T15:40:57.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:57.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:56 smithi192 ceph-mon[130806]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:57.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:56 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:40:57.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:56 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:40:57.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:56 smithi134 ceph-mon[130447]: osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] boot 2023-08-11T15:40:57.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:56 smithi134 ceph-mon[130447]: osdmap e34: 5 total, 5 up, 5 in 2023-08-11T15:40:57.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:57.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:56 smithi134 ceph-mon[130447]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:57.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:56 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:40:57.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:56 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:40:57.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:56 smithi032 ceph-mon[97103]: osd.4 [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] boot 2023-08-11T15:40:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:56 smithi032 ceph-mon[97103]: osdmap e34: 5 total, 5 up, 5 in 2023-08-11T15:40:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:40:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:56 smithi032 ceph-mon[97103]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T15:40:58.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:57 smithi192 ceph-mon[130806]: osdmap e35: 5 total, 5 up, 5 in 2023-08-11T15:40:58.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:57 smithi192 ceph-mon[130806]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:58.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:58.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:58.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:58.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:57 smithi134 ceph-mon[130447]: osdmap e35: 5 total, 5 up, 5 in 2023-08-11T15:40:58.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:57 smithi134 ceph-mon[130447]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:58.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:58.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:58.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:57 smithi032 ceph-mon[97103]: osdmap e35: 5 total, 5 up, 5 in 2023-08-11T15:40:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:57 smithi032 ceph-mon[97103]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:40:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:40:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:40:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:40:59.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:58 smithi134 ceph-mon[130447]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:40:59.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:58 smithi032 ceph-mon[97103]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:40:59.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:58 smithi192 ceph-mon[130806]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:00.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:59 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/3824478850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]: dispatch 2023-08-11T15:41:00.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:59 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]: dispatch 2023-08-11T15:41:00.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:59 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]': finished 2023-08-11T15:41:00.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:59 smithi134 ceph-mon[130447]: osdmap e36: 6 total, 5 up, 6 in 2023-08-11T15:41:00.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:40:59 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:00.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:59 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/3824478850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]: dispatch 2023-08-11T15:41:00.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:59 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]: dispatch 2023-08-11T15:41:00.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:59 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]': finished 2023-08-11T15:41:00.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:59 smithi032 ceph-mon[97103]: osdmap e36: 6 total, 5 up, 6 in 2023-08-11T15:41:00.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:40:59 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:00.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:59 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/3824478850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]: dispatch 2023-08-11T15:41:00.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:59 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]: dispatch 2023-08-11T15:41:00.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:59 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfb4141a-f600-40c2-a097-ee05482183ac"}]': finished 2023-08-11T15:41:00.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:59 smithi192 ceph-mon[130806]: osdmap e36: 6 total, 5 up, 6 in 2023-08-11T15:41:00.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:40:59 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:01.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:00 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/3928490325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:01.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:00 smithi134 ceph-mon[130447]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:01.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:00 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/3928490325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:01.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:00 smithi032 ceph-mon[97103]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:01.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:00 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/3928490325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:01.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:00 smithi192 ceph-mon[130806]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:03.026 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:02 smithi192 ceph-mon[130806]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:03.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:02 smithi032 ceph-mon[97103]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:03.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:02 smithi134 ceph-mon[130447]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:05.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:05 smithi192 ceph-mon[130806]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:05.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:05 smithi134 ceph-mon[130447]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:05.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:05 smithi032 ceph-mon[97103]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:06.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-11T15:41:06.402 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:06 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:06.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-11T15:41:06.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:06 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:06.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-11T15:41:06.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:06 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:07.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:07 smithi192 ceph-mon[130806]: Deploying daemon osd.5 on smithi192 2023-08-11T15:41:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:07 smithi192 ceph-mon[130806]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:07 smithi134 ceph-mon[130447]: Deploying daemon osd.5 on smithi192 2023-08-11T15:41:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:07 smithi134 ceph-mon[130447]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:07.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:07 smithi032 ceph-mon[97103]: Deploying daemon osd.5 on smithi192 2023-08-11T15:41:07.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:07 smithi032 ceph-mon[97103]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:09.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:09 smithi192 ceph-mon[130806]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:09.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:09 smithi134 ceph-mon[130447]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:09.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:09 smithi032 ceph-mon[97103]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:10 smithi192 ceph-mon[130806]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:11.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:11.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:10 smithi032 ceph-mon[97103]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:11.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:11.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:11.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:11.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:10 smithi134 ceph-mon[130447]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:12.461 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.459+0000 7f28e77fe700 1 -- 172.21.15.192:0/3156212137 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f28f8069440 con 0x7f28e009ffd0 2023-08-11T15:41:12.462 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2023-08-11T15:41:12.465 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f28e009ffd0 msgr2=0x7f28e00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:12.465 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f28e009ffd0 0x7f28e00a2490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f28e80036f0 tx=0x7f28e8003680 comp rx=0 tx=0).stop 2023-08-11T15:41:12.465 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f8100ab0 msgr2=0x7f28f8101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:12.465 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f8100ab0 0x7f28f8101e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00b530 tx=0x7f28ec00b840 comp rx=0 tx=0).stop 2023-08-11T15:41:12.466 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 shutdown_connections 2023-08-11T15:41:12.466 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f28e009ffd0 0x7f28e00a2490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:12.466 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f80ff8b0 0x7f28f8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:12.466 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28f8101450 0x7f28f81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:12.466 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 --2- 172.21.15.192:0/3156212137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f8100ab0 0x7f28f8101e90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:12.466 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 >> 172.21.15.192:0/3156212137 conn(0x7f28f80fae30 msgr2=0x7f28f8068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:41:12.466 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 shutdown_connections 2023-08-11T15:41:12.467 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:41:12.462+0000 7f28fcf0d700 1 -- 172.21.15.192:0/3156212137 wait complete. 2023-08-11T15:41:12.867 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.5.service 2023-08-11T15:41:12.869 INFO:tasks.cephadm:Deploying osd.6 on smithi032 with /dev/vg_nvme/lv_2... 2023-08-11T15:41:12.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-11T15:41:12.913 INFO:journalctl@ceph.osd.5.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:41:13.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:13.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:13.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:13.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:13.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:13.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:13.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:13.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:13.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:41:14.078 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:41:13 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:41:13.710+0000 7f7ad92d2640 -1 osd.5 0 log_to_monitors true 2023-08-11T15:41:14.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:13 smithi192 ceph-mon[130806]: from='osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T15:41:14.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:13 smithi192 ceph-mon[130806]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T15:41:14.192 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:13 smithi032 ceph-mon[97103]: from='osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T15:41:14.193 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:13 smithi032 ceph-mon[97103]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T15:41:14.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:13 smithi134 ceph-mon[130447]: from='osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T15:41:14.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:13 smithi134 ceph-mon[130447]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T15:41:15.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:14 smithi134 ceph-mon[130447]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-11T15:41:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:14 smithi134 ceph-mon[130447]: osdmap e37: 6 total, 5 up, 6 in 2023-08-11T15:41:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:14 smithi134 ceph-mon[130447]: from='osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:41:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:14 smithi134 ceph-mon[130447]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:41:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:14 smithi134 ceph-mon[130447]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:14 smithi032 ceph-mon[97103]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-11T15:41:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:14 smithi032 ceph-mon[97103]: osdmap e37: 6 total, 5 up, 6 in 2023-08-11T15:41:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:14 smithi032 ceph-mon[97103]: from='osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:41:15.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:14 smithi032 ceph-mon[97103]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:41:15.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:14 smithi032 ceph-mon[97103]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:15.333 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:41:14.893+0000 7f7ace94a700 -1 osd.5 0 waiting for initial osdmap 2023-08-11T15:41:15.334 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:41:14.900+0000 7f7ac8f3c700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:41:15.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-mon[130806]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-11T15:41:15.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-mon[130806]: osdmap e37: 6 total, 5 up, 6 in 2023-08-11T15:41:15.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:15.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-mon[130806]: from='osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:41:15.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-mon[130806]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:41:15.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:14 smithi192 ceph-mon[130806]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:15.600 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-11T15:41:15.601 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-11T15:41:15.601 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-08-11T15:41:15.601 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-08-11T15:41:15.601 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0305277 s, 343 MB/s 2023-08-11T15:41:15.601 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 2023-08-11T15:41:15.602 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-08-11T15:41:16.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi032:vg_nvme/lv_2 2023-08-11T15:41:16.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:41:16.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: osdmap e38: 6 total, 5 up, 6 in 2023-08-11T15:41:16.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:16.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:16.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi192 2023-08-11T15:41:16.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:16.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:16.176 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi192 to 8584M 2023-08-11T15:41:16.176 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.176 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:16.176 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:16.176 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:41:16.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: osdmap e38: 6 total, 5 up, 6 in 2023-08-11T15:41:16.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:16.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:16.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi192 2023-08-11T15:41:16.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:16.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:16.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi192 to 8584M 2023-08-11T15:41:16.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:16.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:16.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:41:16.347 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: osdmap e38: 6 total, 5 up, 6 in 2023-08-11T15:41:16.347 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:16.347 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:16.348 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi192 2023-08-11T15:41:16.348 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.348 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.348 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:16.348 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:16.348 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi192 to 8584M 2023-08-11T15:41:16.349 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.349 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:16.349 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:16.349 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:16.842 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:41:17.163 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:16 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:41:17.163 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:16 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:41:17.163 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:16 smithi032 ceph-mon[97103]: osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] boot 2023-08-11T15:41:17.164 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:16 smithi032 ceph-mon[97103]: osdmap e39: 6 total, 6 up, 6 in 2023-08-11T15:41:17.164 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:17.164 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:16 smithi032 ceph-mon[97103]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:17.165 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:16 smithi032 ceph-mon[97103]: osdmap e40: 6 total, 6 up, 6 in 2023-08-11T15:41:17.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:16 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:41:17.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:16 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:41:17.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:16 smithi134 ceph-mon[130447]: osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] boot 2023-08-11T15:41:17.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:16 smithi134 ceph-mon[130447]: osdmap e39: 6 total, 6 up, 6 in 2023-08-11T15:41:17.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:17.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:16 smithi134 ceph-mon[130447]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:17.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:16 smithi134 ceph-mon[130447]: osdmap e40: 6 total, 6 up, 6 in 2023-08-11T15:41:17.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:16 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:41:17.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:16 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:41:17.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:16 smithi192 ceph-mon[130806]: osd.5 [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] boot 2023-08-11T15:41:17.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:16 smithi192 ceph-mon[130806]: osdmap e39: 6 total, 6 up, 6 in 2023-08-11T15:41:17.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:41:17.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:16 smithi192 ceph-mon[130806]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T15:41:17.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:16 smithi192 ceph-mon[130806]: osdmap e40: 6 total, 6 up, 6 in 2023-08-11T15:41:18.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 -- 172.21.15.32:0/3711614242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c101bb0 msgr2=0x7eff1c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 --2- 172.21.15.32:0/3711614242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c101bb0 0x7eff1c102010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7eff080099c0 tx=0x7eff08009cd0 comp rx=0 tx=0).stop 2023-08-11T15:41:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 -- 172.21.15.32:0/3711614242 shutdown_connections 2023-08-11T15:41:18.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 --2- 172.21.15.32:0/3711614242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1c102550 0x7eff1c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:18.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 --2- 172.21.15.32:0/3711614242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff1c1009b0 0x7eff1c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:18.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 --2- 172.21.15.32:0/3711614242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c101bb0 0x7eff1c102010 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:18.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 -- 172.21.15.32:0/3711614242 >> 172.21.15.32:0/3711614242 conn(0x7eff1c0fbf10 msgr2=0x7eff1c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:41:18.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 -- 172.21.15.32:0/3711614242 shutdown_connections 2023-08-11T15:41:18.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.381+0000 7eff22fec700 1 -- 172.21.15.32:0/3711614242 wait complete. 2023-08-11T15:41:18.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.382+0000 7eff22fec700 1 Processor -- start 2023-08-11T15:41:18.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.382+0000 7eff22fec700 1 -- start start 2023-08-11T15:41:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff22fec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c1009b0 0x7eff1c0746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff22fec700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff1c101bb0 0x7eff1c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff22fec700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1c102550 0x7eff1c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff22fec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0739d0 con 0x7eff1c1009b0 2023-08-11T15:41:18.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff22fec700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c073b40 con 0x7eff1c101bb0 2023-08-11T15:41:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff22fec700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1a06c0 con 0x7eff1c102550 2023-08-11T15:41:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff20d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c1009b0 0x7eff1c0746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff20d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c1009b0 0x7eff1c0746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58328/0 (socket says 172.21.15.32:58328) 2023-08-11T15:41:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff20d88700 1 -- 172.21.15.32:0/1457869495 learned_addr learned my addr 172.21.15.32:0/1457869495 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:41:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff1bfff700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff1c101bb0 0x7eff1c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.383+0000 7eff21589700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1c102550 0x7eff1c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff20d88700 1 -- 172.21.15.32:0/1457869495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1c102550 msgr2=0x7eff1c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:18.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff20d88700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1c102550 0x7eff1c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:18.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff20d88700 1 -- 172.21.15.32:0/1457869495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff1c101bb0 msgr2=0x7eff1c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:18.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff20d88700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff1c101bb0 0x7eff1c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:18.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff20d88700 1 -- 172.21.15.32:0/1457869495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080095f0 con 0x7eff1c1009b0 2023-08-11T15:41:18.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff20d88700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c1009b0 0x7eff1c0746c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7eff1000d980 tx=0x7eff1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:41:18.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff19ffb700 1 -- 172.21.15.32:0/1457869495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1000f890 con 0x7eff1c1009b0 2023-08-11T15:41:18.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c1a09a0 con 0x7eff1c1009b0 2023-08-11T15:41:18.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff19ffb700 1 -- 172.21.15.32:0/1457869495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff10010460 con 0x7eff1c1009b0 2023-08-11T15:41:18.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff19ffb700 1 -- 172.21.15.32:0/1457869495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff100195c0 con 0x7eff1c1009b0 2023-08-11T15:41:18.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.384+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c1a0ea0 con 0x7eff1c1009b0 2023-08-11T15:41:18.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.385+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c10c1b0 con 0x7eff1c1009b0 2023-08-11T15:41:18.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.387+0000 7eff19ffb700 1 -- 172.21.15.32:0/1457869495 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7eff100105d0 con 0x7eff1c1009b0 2023-08-11T15:41:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.389+0000 7eff19ffb700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff0c09ff50 0x7eff0c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.389+0000 7eff19ffb700 1 -- 172.21.15.32:0/1457869495 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7eff100ca1b0 con 0x7eff1c1009b0 2023-08-11T15:41:18.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.389+0000 7eff1bfff700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff0c09ff50 0x7eff0c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:18.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.390+0000 7eff1bfff700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff0c09ff50 0x7eff0c0a2410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff0800b580 tx=0x7eff08005bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:41:18.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.393+0000 7eff19ffb700 1 -- 172.21.15.32:0/1457869495 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7eff10094d40 con 0x7eff1c1009b0 2023-08-11T15:41:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:18.611+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7eff1c060d60 con 0x7eff0c09ff50 2023-08-11T15:41:19.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:18 smithi032 ceph-mon[97103]: osdmap e41: 6 total, 6 up, 6 in 2023-08-11T15:41:19.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:18 smithi032 ceph-mon[97103]: pgmap v133: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:19.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:18 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:41:19.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:18 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:41:19.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:18 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:19.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:18 smithi192 ceph-mon[130806]: osdmap e41: 6 total, 6 up, 6 in 2023-08-11T15:41:19.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:18 smithi192 ceph-mon[130806]: pgmap v133: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:19.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:18 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:41:19.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:18 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:41:19.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:18 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:19.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:18 smithi134 ceph-mon[130447]: osdmap e41: 6 total, 6 up, 6 in 2023-08-11T15:41:19.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:18 smithi134 ceph-mon[130447]: pgmap v133: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:19.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:18 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:41:19.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:18 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:41:19.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:18 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:20.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:19 smithi032 ceph-mon[97103]: from='client.14328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:41:20.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:19 smithi032 ceph-mon[97103]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-08-11T15:41:20.017 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:19 smithi032 ceph-mon[97103]: osdmap e42: 6 total, 6 up, 6 in 2023-08-11T15:41:20.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:19 smithi192 ceph-mon[130806]: from='client.14328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:41:20.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:19 smithi192 ceph-mon[130806]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-08-11T15:41:20.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:19 smithi192 ceph-mon[130806]: osdmap e42: 6 total, 6 up, 6 in 2023-08-11T15:41:20.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:19 smithi134 ceph-mon[130447]: from='client.14328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:41:20.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:19 smithi134 ceph-mon[130447]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-08-11T15:41:20.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:19 smithi134 ceph-mon[130447]: osdmap e42: 6 total, 6 up, 6 in 2023-08-11T15:41:21.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:20 smithi032 ceph-mon[97103]: osdmap e43: 6 total, 6 up, 6 in 2023-08-11T15:41:21.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:20 smithi032 ceph-mon[97103]: pgmap v136: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:21.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:20 smithi192 ceph-mon[130806]: osdmap e43: 6 total, 6 up, 6 in 2023-08-11T15:41:21.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:20 smithi192 ceph-mon[130806]: pgmap v136: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:21.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:20 smithi134 ceph-mon[130447]: osdmap e43: 6 total, 6 up, 6 in 2023-08-11T15:41:21.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:20 smithi134 ceph-mon[130447]: pgmap v136: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:22.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:21 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2781191711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ba2f452-da4c-4d75-9452-fd60cdcc25a3"}]: dispatch 2023-08-11T15:41:22.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:21 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2781191711' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ba2f452-da4c-4d75-9452-fd60cdcc25a3"}]': finished 2023-08-11T15:41:22.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:21 smithi032 ceph-mon[97103]: osdmap e44: 7 total, 6 up, 7 in 2023-08-11T15:41:22.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:21 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:22.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:21 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2781191711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ba2f452-da4c-4d75-9452-fd60cdcc25a3"}]: dispatch 2023-08-11T15:41:22.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:21 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2781191711' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ba2f452-da4c-4d75-9452-fd60cdcc25a3"}]': finished 2023-08-11T15:41:22.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:21 smithi192 ceph-mon[130806]: osdmap e44: 7 total, 6 up, 7 in 2023-08-11T15:41:22.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:21 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:22.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:21 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2781191711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ba2f452-da4c-4d75-9452-fd60cdcc25a3"}]: dispatch 2023-08-11T15:41:22.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:21 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2781191711' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ba2f452-da4c-4d75-9452-fd60cdcc25a3"}]': finished 2023-08-11T15:41:22.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:21 smithi134 ceph-mon[130447]: osdmap e44: 7 total, 6 up, 7 in 2023-08-11T15:41:22.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:21 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:23.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:22 smithi032 ceph-mon[97103]: pgmap v138: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:23.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:22 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/149555127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:23.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:22 smithi192 ceph-mon[130806]: pgmap v138: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:22 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/149555127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:23.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:22 smithi134 ceph-mon[130447]: pgmap v138: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2023-08-11T15:41:23.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:22 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/149555127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:24.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:24 smithi134 ceph-mon[130447]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-08-11T15:41:24.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:24 smithi134 ceph-mon[130447]: Cluster is now healthy 2023-08-11T15:41:24.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:24 smithi032 ceph-mon[97103]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-08-11T15:41:24.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:24 smithi032 ceph-mon[97103]: Cluster is now healthy 2023-08-11T15:41:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:24 smithi192 ceph-mon[130806]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-08-11T15:41:24.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:24 smithi192 ceph-mon[130806]: Cluster is now healthy 2023-08-11T15:41:25.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:25 smithi134 ceph-mon[130447]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:41:25.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:25 smithi032 ceph-mon[97103]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:41:25.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:25 smithi192 ceph-mon[130806]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:41:27.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:27 smithi032 ceph-mon[97103]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-08-11T15:41:27.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:27 smithi134 ceph-mon[130447]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-08-11T15:41:27.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:27 smithi192 ceph-mon[130806]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-08-11T15:41:29.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:29 smithi134 ceph-mon[130447]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:41:29.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:29 smithi032 ceph-mon[97103]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:41:29.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:29 smithi192 ceph-mon[130806]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:41:30.456 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-11T15:41:30.456 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:30.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:30 smithi032 ceph-mon[97103]: Deploying daemon osd.6 on smithi032 2023-08-11T15:41:30.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-11T15:41:30.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:30.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:30 smithi134 ceph-mon[130447]: Deploying daemon osd.6 on smithi032 2023-08-11T15:41:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-11T15:41:30.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:30.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:30 smithi192 ceph-mon[130806]: Deploying daemon osd.6 on smithi032 2023-08-11T15:41:31.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:31 smithi134 ceph-mon[130447]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:41:31.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:31 smithi032 ceph-mon[97103]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:41:31.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:31 smithi192 ceph-mon[130806]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:41:32.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:32 smithi032 ceph-mon[97103]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-08-11T15:41:33.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:32 smithi192 ceph-mon[130806]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-08-11T15:41:33.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:32 smithi134 ceph-mon[130447]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-08-11T15:41:35.355 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:35 smithi032 ceph-mon[97103]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T15:41:35.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:35 smithi134 ceph-mon[130447]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T15:41:35.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:35 smithi192 ceph-mon[130806]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T15:41:36.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:36.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:36.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:36.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:36.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:36.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:37.282 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:37 smithi032 ceph-mon[97103]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:37.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:37 smithi134 ceph-mon[130447]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:37.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:37 smithi192 ceph-mon[130806]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:38.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.977+0000 7eff19ffb700 1 -- 172.21.15.32:0/1457869495 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff1c060d60 con 0x7eff0c09ff50 2023-08-11T15:41:38.981 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 6 on host 'smithi032' 2023-08-11T15:41:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.979+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff0c09ff50 msgr2=0x7eff0c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.979+0000 7eff22fec700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff0c09ff50 0x7eff0c0a2410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff0800b580 tx=0x7eff08005bc0 comp rx=0 tx=0).stop 2023-08-11T15:41:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.979+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c1009b0 msgr2=0x7eff1c0746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.979+0000 7eff22fec700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c1009b0 0x7eff1c0746c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7eff1000d980 tx=0x7eff1000dc90 comp rx=0 tx=0).stop 2023-08-11T15:41:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 shutdown_connections 2023-08-11T15:41:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff0c09ff50 0x7eff0c0a2410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1c1009b0 0x7eff1c0746c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1c102550 0x7eff1c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 --2- 172.21.15.32:0/1457869495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff1c101bb0 0x7eff1c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 >> 172.21.15.32:0/1457869495 conn(0x7eff1c0fbf10 msgr2=0x7eff1c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:41:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 shutdown_connections 2023-08-11T15:41:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:41:38.980+0000 7eff22fec700 1 -- 172.21.15.32:0/1457869495 wait complete. 2023-08-11T15:41:39.179 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:39.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:39.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:39.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:39.466 DEBUG:teuthology.orchestra.run.smithi032:osd.6> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.6.service 2023-08-11T15:41:39.473 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_2... 2023-08-11T15:41:39.473 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-11T15:41:39.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:39.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:39.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:39.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:39 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:39.554 INFO:journalctl@ceph.osd.6.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:41:39.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:39.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:39 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:39.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:41:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:41:39.558+0000 7fdc15734640 -1 osd.6 0 log_to_monitors true 2023-08-11T15:41:40.049 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:41:40.280 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:40 smithi032 ceph-mon[97103]: from='osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T15:41:40.280 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:40 smithi032 ceph-mon[97103]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T15:41:40.416 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:40 smithi134 ceph-mon[130447]: from='osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T15:41:40.416 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:40 smithi134 ceph-mon[130447]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T15:41:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:40 smithi192 ceph-mon[130806]: from='osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T15:41:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:40 smithi192 ceph-mon[130806]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T15:41:41.123 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:41:41.109+0000 7fdc0adac700 -1 osd.6 0 waiting for initial osdmap 2023-08-11T15:41:41.123 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:41:41.119+0000 7fdc02b99700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:41:41.123 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-mon[97103]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:41.123 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-mon[97103]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-11T15:41:41.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-mon[97103]: osdmap e45: 7 total, 6 up, 7 in 2023-08-11T15:41:41.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:41.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-mon[97103]: from='osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:41:41.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:41 smithi032 ceph-mon[97103]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:41:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:41 smithi134 ceph-mon[130447]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:41 smithi134 ceph-mon[130447]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-11T15:41:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:41 smithi134 ceph-mon[130447]: osdmap e45: 7 total, 6 up, 7 in 2023-08-11T15:41:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:41 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:41 smithi134 ceph-mon[130447]: from='osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:41:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:41 smithi134 ceph-mon[130447]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:41:41.522 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-11T15:41:41.528 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-11T15:41:41.528 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-08-11T15:41:41.528 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-08-11T15:41:41.528 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110587 s, 948 MB/s 2023-08-11T15:41:41.528 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-08-11T15:41:41.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:41 smithi192 ceph-mon[130806]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:41.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:41 smithi192 ceph-mon[130806]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-11T15:41:41.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:41 smithi192 ceph-mon[130806]: osdmap e45: 7 total, 6 up, 7 in 2023-08-11T15:41:41.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:41 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:41.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:41 smithi192 ceph-mon[130806]: from='osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:41:41.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:41 smithi192 ceph-mon[130806]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:41:41.889 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2023-08-11T15:41:42.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:42 smithi134 ceph-mon[130447]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:41:42.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:42 smithi134 ceph-mon[130447]: osdmap e46: 7 total, 6 up, 7 in 2023-08-11T15:41:42.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:42.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:42.440 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:41:42.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:42 smithi032 ceph-mon[97103]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:41:42.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:42 smithi032 ceph-mon[97103]: osdmap e46: 7 total, 6 up, 7 in 2023-08-11T15:41:42.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:42.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:42 smithi192 ceph-mon[130806]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:41:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:42 smithi192 ceph-mon[130806]: osdmap e46: 7 total, 6 up, 7 in 2023-08-11T15:41:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:43.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:41:43.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:41:43.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:43.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] boot 2023-08-11T15:41:43.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: osdmap e47: 7 total, 7 up, 7 in 2023-08-11T15:41:43.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:43.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi032 2023-08-11T15:41:43.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi032 to 5757M 2023-08-11T15:41:43.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.219 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:43.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:43.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:43 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:41:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:41:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] boot 2023-08-11T15:41:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: osdmap e47: 7 total, 7 up, 7 in 2023-08-11T15:41:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi032 2023-08-11T15:41:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi032 to 5757M 2023-08-11T15:41:43.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:43.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:43.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:43 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:41:43.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:41:43.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T15:41:43.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: osd.6 [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] boot 2023-08-11T15:41:43.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: osdmap e47: 7 total, 7 up, 7 in 2023-08-11T15:41:43.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:41:43.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi032 2023-08-11T15:41:43.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:41:43.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi032 to 5757M 2023-08-11T15:41:43.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:43.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:41:43.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:43 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:43.778 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.776+0000 7f9c6a88a700 1 -- 172.21.15.134:0/400486496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c64100ef0 msgr2=0x7f9c64101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:43.779 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.776+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/400486496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c64100ef0 0x7f9c64101310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009990 tx=0x7f9c54009ca0 comp rx=0 tx=0).stop 2023-08-11T15:41:43.779 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.776+0000 7f9c6a88a700 1 -- 172.21.15.134:0/400486496 shutdown_connections 2023-08-11T15:41:43.779 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.776+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/400486496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c64100540 0x7f9c64107e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:43.779 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.776+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/400486496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c64100ef0 0x7f9c64101310 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:43.779 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.776+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/400486496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c640ffba0 0x7f9c64100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:43.779 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.776+0000 7f9c6a88a700 1 -- 172.21.15.134:0/400486496 >> 172.21.15.134:0/400486496 conn(0x7f9c640fa630 msgr2=0x7f9c640fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:41:43.780 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 -- 172.21.15.134:0/400486496 shutdown_connections 2023-08-11T15:41:43.780 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 -- 172.21.15.134:0/400486496 wait complete. 2023-08-11T15:41:43.780 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 Processor -- start 2023-08-11T15:41:43.780 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 -- start start 2023-08-11T15:41:43.781 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c640ffba0 0x7f9c64195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:43.782 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c64100540 0x7f9c64195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:43.782 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c64100ef0 0x7f9c6419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:43.782 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6419d110 con 0x7f9c64100ef0 2023-08-11T15:41:43.782 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6419d250 con 0x7f9c640ffba0 2023-08-11T15:41:43.782 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.777+0000 7f9c6a88a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6419d3e0 con 0x7f9c64100540 2023-08-11T15:41:43.782 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c640ffba0 0x7f9c64195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:43.782 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c640ffba0 0x7f9c64195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53432/0 (socket says 172.21.15.134:53432) 2023-08-11T15:41:43.783 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 -- 172.21.15.134:0/572015293 learned_addr learned my addr 172.21.15.134:0/572015293 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-08-11T15:41:43.783 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c637fe700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c64100540 0x7f9c64195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:43.783 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c68e27700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c64100ef0 0x7f9c6419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:43.783 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 -- 172.21.15.134:0/572015293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c64100540 msgr2=0x7f9c64195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:43.783 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c64100540 0x7f9c64195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:43.783 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 -- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c64100ef0 msgr2=0x7f9c6419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:43.783 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c64100ef0 0x7f9c6419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:43.784 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 -- 172.21.15.134:0/572015293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c540095f0 con 0x7f9c640ffba0 2023-08-11T15:41:43.784 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c68e27700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c64100ef0 0x7f9c6419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:41:43.784 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c63fff700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c640ffba0 0x7f9c64195580 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9c540095c0 tx=0x7f9c5400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:41:43.785 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c617fa700 1 -- 172.21.15.134:0/572015293 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5401d070 con 0x7f9c640ffba0 2023-08-11T15:41:43.785 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c617fa700 1 -- 172.21.15.134:0/572015293 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c54022470 con 0x7f9c640ffba0 2023-08-11T15:41:43.785 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6419d660 con 0x7f9c640ffba0 2023-08-11T15:41:43.785 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c617fa700 1 -- 172.21.15.134:0/572015293 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54017650 con 0x7f9c640ffba0 2023-08-11T15:41:43.785 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.778+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6419da70 con 0x7f9c640ffba0 2023-08-11T15:41:43.786 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.779+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c64106f00 con 0x7f9c640ffba0 2023-08-11T15:41:43.786 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.780+0000 7f9c617fa700 1 -- 172.21.15.134:0/572015293 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f9c54022870 con 0x7f9c640ffba0 2023-08-11T15:41:43.787 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.781+0000 7f9c617fa700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c4c09ff50 0x7f9c4c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:41:43.787 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.781+0000 7f9c617fa700 1 -- 172.21.15.134:0/572015293 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f9c540cb820 con 0x7f9c640ffba0 2023-08-11T15:41:43.787 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.781+0000 7f9c637fe700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c4c09ff50 0x7f9c4c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:41:43.787 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.781+0000 7f9c637fe700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c4c09ff50 0x7f9c4c0a2410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c58005950 tx=0x7f9c5800a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:41:43.787 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.785+0000 7f9c617fa700 1 -- 172.21.15.134:0/572015293 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9c54096340 con 0x7f9c640ffba0 2023-08-11T15:41:43.980 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:43.979+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c6402ce20 con 0x7f9c4c09ff50 2023-08-11T15:41:44.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:44 smithi134 ceph-mon[130447]: osdmap e48: 7 total, 7 up, 7 in 2023-08-11T15:41:44.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:44 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:41:44.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:44 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:41:44.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:44 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:44 smithi032 ceph-mon[97103]: osdmap e48: 7 total, 7 up, 7 in 2023-08-11T15:41:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:44 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:41:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:44 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:41:44.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:44 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:44.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:44 smithi192 ceph-mon[130806]: osdmap e48: 7 total, 7 up, 7 in 2023-08-11T15:41:44.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:44 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:41:44.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:44 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:41:44.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:44 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:45 smithi134 ceph-mon[130447]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:41:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:45 smithi134 ceph-mon[130447]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:45 smithi032 ceph-mon[97103]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:41:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:45 smithi032 ceph-mon[97103]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:45 smithi192 ceph-mon[130806]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:41:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:45 smithi192 ceph-mon[130806]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:47 smithi134 ceph-mon[130447]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:47 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/1532540847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]: dispatch 2023-08-11T15:41:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:47 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]: dispatch 2023-08-11T15:41:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:47 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]': finished 2023-08-11T15:41:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:47 smithi134 ceph-mon[130447]: osdmap e49: 8 total, 7 up, 8 in 2023-08-11T15:41:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:41:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:47 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/417272510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:47.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:47 smithi032 ceph-mon[97103]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:47.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:47 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/1532540847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]: dispatch 2023-08-11T15:41:47.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:47 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]: dispatch 2023-08-11T15:41:47.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:47 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]': finished 2023-08-11T15:41:47.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:47 smithi032 ceph-mon[97103]: osdmap e49: 8 total, 7 up, 8 in 2023-08-11T15:41:47.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:41:47.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:47 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/417272510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:47.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:47 smithi192 ceph-mon[130806]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:47 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/1532540847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]: dispatch 2023-08-11T15:41:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:47 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]: dispatch 2023-08-11T15:41:47.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:47 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12bf8d04-54c5-4aa8-ae64-14779ba45002"}]': finished 2023-08-11T15:41:47.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:47 smithi192 ceph-mon[130806]: osdmap e49: 8 total, 7 up, 8 in 2023-08-11T15:41:47.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:41:47.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:47 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/417272510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:41:49.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:49 smithi134 ceph-mon[130447]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:49.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:49 smithi032 ceph-mon[97103]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:49.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:49 smithi192 ceph-mon[130806]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:51.395 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:51 smithi134 ceph-mon[130447]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:51 smithi032 ceph-mon[97103]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:51.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:51 smithi192 ceph-mon[130806]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:52.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:52 smithi134 ceph-mon[130447]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:52.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-11T15:41:52.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:53.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:52 smithi032 ceph-mon[97103]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:53.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-11T15:41:53.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:53.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:52 smithi192 ceph-mon[130806]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:53.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-11T15:41:53.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:41:53.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:53 smithi134 ceph-mon[130447]: Deploying daemon osd.7 on smithi134 2023-08-11T15:41:54.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:53 smithi032 ceph-mon[97103]: Deploying daemon osd.7 on smithi134 2023-08-11T15:41:54.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:53 smithi192 ceph-mon[130806]: Deploying daemon osd.7 on smithi134 2023-08-11T15:41:54.916 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:54 smithi134 ceph-mon[130447]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:55.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:54 smithi032 ceph-mon[97103]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:55.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:54 smithi192 ceph-mon[130806]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:57 smithi134 ceph-mon[130447]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:57 smithi032 ceph-mon[97103]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:57.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:57 smithi192 ceph-mon[130806]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:58.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:58.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:58.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:58.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:58.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:58.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:41:58.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:41:59.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:41:59 smithi134 ceph-mon[130447]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:59.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:41:59 smithi032 ceph-mon[97103]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:59.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:41:59 smithi192 ceph-mon[130806]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:41:59.995 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2023-08-11T15:41:59.996 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.993+0000 7f9c617fa700 1 -- 172.21.15.134:0/572015293 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9c6402ce20 con 0x7f9c4c09ff50 2023-08-11T15:41:59.998 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c4c09ff50 msgr2=0x7f9c4c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:59.998 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c4c09ff50 0x7f9c4c0a2410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c58005950 tx=0x7f9c5800a300 comp rx=0 tx=0).stop 2023-08-11T15:41:59.998 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c640ffba0 msgr2=0x7f9c64195580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:41:59.998 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c640ffba0 0x7f9c64195580 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9c540095c0 tx=0x7f9c5400f690 comp rx=0 tx=0).stop 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 shutdown_connections 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c4c09ff50 0x7f9c4c0a2410 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c64100ef0 0x7f9c6419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c640ffba0 0x7f9c64195580 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 --2- 172.21.15.134:0/572015293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c64100540 0x7f9c64195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.996+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 >> 172.21.15.134:0/572015293 conn(0x7f9c640fa630 msgr2=0x7f9c64104040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.997+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 shutdown_connections 2023-08-11T15:41:59.999 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:41:59.997+0000 7f9c6a88a700 1 -- 172.21.15.134:0/572015293 wait complete. 2023-08-11T15:42:00.429 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.7.service 2023-08-11T15:42:00.432 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2023-08-11T15:42:00.432 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-11T15:42:00.463 INFO:journalctl@ceph.osd.7.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:42:00.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:00 smithi134 ceph-mon[130447]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:00 smithi192 ceph-mon[130806]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:00.994 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:42:01.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:01.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:01.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:01.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:01.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:01.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:01.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:01.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:01.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:00 smithi032 ceph-mon[97103]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:01.177 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:42:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:42:01.096+0000 7f291611e640 -1 osd.7 0 log_to_monitors true 2023-08-11T15:42:02.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:01 smithi032 ceph-mon[97103]: from='osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T15:42:02.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:01 smithi032 ceph-mon[97103]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T15:42:02.327 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:01 smithi134 ceph-mon[130447]: from='osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T15:42:02.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:01 smithi134 ceph-mon[130447]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T15:42:02.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:01 smithi192 ceph-mon[130806]: from='osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T15:42:02.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:01 smithi192 ceph-mon[130806]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T15:42:02.360 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-11T15:42:02.361 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-11T15:42:02.361 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-08-11T15:42:02.361 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-08-11T15:42:02.362 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126425 s, 829 MB/s 2023-08-11T15:42:02.362 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-08-11T15:42:02.750 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2023-08-11T15:42:03.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-11T15:42:03.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: osdmap e50: 8 total, 7 up, 8 in 2023-08-11T15:42:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:42:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:42:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:03.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:03 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.307 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:42:03.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-11T15:42:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: osdmap e50: 8 total, 7 up, 8 in 2023-08-11T15:42:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:42:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:42:03.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:03.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:03.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:03 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.471 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:42:03.003+0000 7f290b796700 -1 osd.7 0 waiting for initial osdmap 2023-08-11T15:42:03.471 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:42:03.014+0000 7f2905587700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:42:03.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-11T15:42:03.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: osdmap e50: 8 total, 7 up, 8 in 2023-08-11T15:42:03.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:42:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:42:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:03.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:03.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:03.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:03.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:03 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:04.261 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:04 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi134 2023-08-11T15:42:04.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:04 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi134 to 5722M 2023-08-11T15:42:04.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:04 smithi192 ceph-mon[130806]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:42:04.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:04 smithi192 ceph-mon[130806]: osdmap e51: 8 total, 7 up, 8 in 2023-08-11T15:42:04.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:04.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:04 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi134 2023-08-11T15:42:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:04 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi134 to 5722M 2023-08-11T15:42:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:04 smithi032 ceph-mon[97103]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:42:04.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:04 smithi032 ceph-mon[97103]: osdmap e51: 8 total, 7 up, 8 in 2023-08-11T15:42:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:04 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi134 2023-08-11T15:42:04.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:04 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi134 to 5722M 2023-08-11T15:42:04.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:04 smithi134 ceph-mon[130447]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:42:04.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:04 smithi134 ceph-mon[130447]: osdmap e51: 8 total, 7 up, 8 in 2023-08-11T15:42:04.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:04.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:04.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 -- 172.21.15.192:0/1401059700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80ff8b0 msgr2=0x7f1ae80ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:04.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 --2- 172.21.15.192:0/1401059700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80ff8b0 0x7f1ae80ffcd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009990 tx=0x7f1adc009ca0 comp rx=0 tx=0).stop 2023-08-11T15:42:04.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 -- 172.21.15.192:0/1401059700 shutdown_connections 2023-08-11T15:42:04.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 --2- 172.21.15.192:0/1401059700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8101450 0x7f1ae8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:04.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 --2- 172.21.15.192:0/1401059700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80ff8b0 0x7f1ae80ffcd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:04.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 --2- 172.21.15.192:0/1401059700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8100ab0 0x7f1ae8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:04.477 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 -- 172.21.15.192:0/1401059700 >> 172.21.15.192:0/1401059700 conn(0x7f1ae80fae30 msgr2=0x7f1ae80fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:42:04.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 -- 172.21.15.192:0/1401059700 shutdown_connections 2023-08-11T15:42:04.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.470+0000 7f1aede64700 1 -- 172.21.15.192:0/1401059700 wait complete. 2023-08-11T15:42:04.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 Processor -- start 2023-08-11T15:42:04.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 -- start start 2023-08-11T15:42:04.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ff8b0 0x7f1ae8195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:04.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae8100ab0 0x7f1ae8195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:04.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8101450 0x7f1ae819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:04.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae819d190 con 0x7f1ae80ff8b0 2023-08-11T15:42:04.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae819d2d0 con 0x7f1ae8101450 2023-08-11T15:42:04.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1aede64700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae819d460 con 0x7f1ae8100ab0 2023-08-11T15:42:04.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae8100ab0 0x7f1ae8195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:04.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae8100ab0 0x7f1ae8195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57636/0 (socket says 172.21.15.192:57636) 2023-08-11T15:42:04.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 -- 172.21.15.192:0/1549018724 learned_addr learned my addr 172.21.15.192:0/1549018724 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:42:04.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae77fe700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ff8b0 0x7f1ae8195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:04.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae7fff700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8101450 0x7f1ae819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:04.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 -- 172.21.15.192:0/1549018724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8101450 msgr2=0x7f1ae819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:04.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8101450 0x7f1ae819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:04.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 -- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ff8b0 msgr2=0x7f1ae8195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:04.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ff8b0 0x7f1ae8195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:04.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae6ffd700 1 -- 172.21.15.192:0/1549018724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0095f0 con 0x7f1ae8100ab0 2023-08-11T15:42:04.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae77fe700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ff8b0 0x7f1ae8195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.471+0000 7f1ae7fff700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8101450 0x7f1ae819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.472+0000 7f1ae6ffd700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae8100ab0 0x7f1ae8195b40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c950 tx=0x7f1ad800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.472+0000 7f1ae4ff9700 1 -- 172.21.15.192:0/1549018724 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8007ac0 con 0x7f1ae8100ab0 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.472+0000 7f1ae4ff9700 1 -- 172.21.15.192:0/1549018724 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad8004cf0 con 0x7f1ae8100ab0 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.472+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae819d740 con 0x7f1ae8100ab0 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.472+0000 7f1ae4ff9700 1 -- 172.21.15.192:0/1549018724 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8005550 con 0x7f1ae8100ab0 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.472+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae819dd10 con 0x7f1ae8100ab0 2023-08-11T15:42:04.481 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.473+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae8106f00 con 0x7f1ae8100ab0 2023-08-11T15:42:04.482 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.474+0000 7f1ae4ff9700 1 -- 172.21.15.192:0/1549018724 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f1ad8004750 con 0x7f1ae8100ab0 2023-08-11T15:42:04.483 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.474+0000 7f1ae4ff9700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1ad009ffd0 0x7f1ad00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:04.483 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.474+0000 7f1ae4ff9700 1 -- 172.21.15.192:0/1549018724 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f1ad80ca6b0 con 0x7f1ae8100ab0 2023-08-11T15:42:04.483 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.475+0000 7f1ae77fe700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1ad009ffd0 0x7f1ad00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:04.483 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.475+0000 7f1ae77fe700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1ad009ffd0 0x7f1ad00a2490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0095c0 tx=0x7f1adc009510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:42:04.483 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.478+0000 7f1ae4ff9700 1 -- 172.21.15.192:0/1549018724 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1ad8095000 con 0x7f1ae8100ab0 2023-08-11T15:42:04.677 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:04.674+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ae802cde0 con 0x7f1ad009ffd0 2023-08-11T15:42:05.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:42:05.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:42:05.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:05.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] boot 2023-08-11T15:42:05.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: osdmap e52: 8 total, 8 up, 8 in 2023-08-11T15:42:05.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:05.306 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:05.306 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:05.306 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:05.307 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:05.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:42:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:42:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] boot 2023-08-11T15:42:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: osdmap e52: 8 total, 8 up, 8 in 2023-08-11T15:42:05.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:05.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:05.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:05.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:05.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:05.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:42:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:42:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: osd.7 [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] boot 2023-08-11T15:42:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: osdmap e52: 8 total, 8 up, 8 in 2023-08-11T15:42:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:42:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-08-11T15:42:05.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:05.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:05.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:06.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:06 smithi192 ceph-mon[130806]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:06.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:06 smithi192 ceph-mon[130806]: osdmap e53: 8 total, 8 up, 8 in 2023-08-11T15:42:06.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:06 smithi032 ceph-mon[97103]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:06.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:06 smithi032 ceph-mon[97103]: osdmap e53: 8 total, 8 up, 8 in 2023-08-11T15:42:06.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:06 smithi134 ceph-mon[130447]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:06.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:06 smithi134 ceph-mon[130447]: osdmap e53: 8 total, 8 up, 8 in 2023-08-11T15:42:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: osdmap e54: 8 total, 8 up, 8 in 2023-08-11T15:42:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: osdmap e55: 8 total, 8 up, 8 in 2023-08-11T15:42:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/33508749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]: dispatch 2023-08-11T15:42:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]: dispatch 2023-08-11T15:42:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]': finished 2023-08-11T15:42:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: osdmap e56: 9 total, 8 up, 9 in 2023-08-11T15:42:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:07.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: osdmap e54: 8 total, 8 up, 8 in 2023-08-11T15:42:07.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: osdmap e55: 8 total, 8 up, 8 in 2023-08-11T15:42:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/33508749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]: dispatch 2023-08-11T15:42:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]: dispatch 2023-08-11T15:42:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]': finished 2023-08-11T15:42:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: osdmap e56: 9 total, 8 up, 9 in 2023-08-11T15:42:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:07.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: osdmap e54: 8 total, 8 up, 8 in 2023-08-11T15:42:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: osdmap e55: 8 total, 8 up, 8 in 2023-08-11T15:42:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/33508749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]: dispatch 2023-08-11T15:42:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]: dispatch 2023-08-11T15:42:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9797fa6-65ca-40f5-aa93-b4120c4400ce"}]': finished 2023-08-11T15:42:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: osdmap e56: 9 total, 8 up, 9 in 2023-08-11T15:42:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:08 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/1049435737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:08 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/1049435737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:08.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:08 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/1049435737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:09.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:09 smithi032 ceph-mon[97103]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:09.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:09 smithi192 ceph-mon[130806]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:09.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:09 smithi134 ceph-mon[130447]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:11.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:11 smithi192 ceph-mon[130806]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:11.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:11 smithi134 ceph-mon[130447]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:11.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:11 smithi032 ceph-mon[97103]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:13.021 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:12 smithi192 ceph-mon[130806]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:13.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:12 smithi032 ceph-mon[97103]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:13.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:12 smithi134 ceph-mon[130447]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:14.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-11T15:42:14.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:14.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-11T15:42:14.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:14.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-11T15:42:14.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:15.016 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:14 smithi192 ceph-mon[130806]: Deploying daemon osd.8 on smithi192 2023-08-11T15:42:15.017 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:14 smithi192 ceph-mon[130806]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:15.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:14 smithi032 ceph-mon[97103]: Deploying daemon osd.8 on smithi192 2023-08-11T15:42:15.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:14 smithi032 ceph-mon[97103]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:15.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:14 smithi134 ceph-mon[130447]: Deploying daemon osd.8 on smithi192 2023-08-11T15:42:15.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:14 smithi134 ceph-mon[130447]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:17.412 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:17 smithi192 ceph-mon[130806]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:17.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:17 smithi134 ceph-mon[130447]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:17.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:17 smithi032 ceph-mon[97103]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:18.653 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:18 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:18.654 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:18 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:18.654 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:18 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:18.654 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:18 smithi192 ceph-mon[130806]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:18.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:18 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:18.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:18 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:18.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:18 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:18.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:18 smithi032 ceph-mon[97103]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:18.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:18 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:18.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:18 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:18.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:18 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:18.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:18 smithi134 ceph-mon[130447]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:20.087 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.086+0000 7f1ae4ff9700 1 -- 172.21.15.192:0/1549018724 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1ae802cde0 con 0x7f1ad009ffd0 2023-08-11T15:42:20.088 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2023-08-11T15:42:20.090 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1ad009ffd0 msgr2=0x7f1ad00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:20.090 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1ad009ffd0 0x7f1ad00a2490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0095c0 tx=0x7f1adc009510 comp rx=0 tx=0).stop 2023-08-11T15:42:20.090 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae8100ab0 msgr2=0x7f1ae8195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:20.090 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae8100ab0 0x7f1ae8195b40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c950 tx=0x7f1ad800cd10 comp rx=0 tx=0).stop 2023-08-11T15:42:20.090 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 shutdown_connections 2023-08-11T15:42:20.091 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1ad009ffd0 0x7f1ad00a2490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:20.091 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae80ff8b0 0x7f1ae8195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:20.091 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8101450 0x7f1ae819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:20.091 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 --2- 172.21.15.192:0/1549018724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae8100ab0 0x7f1ae8195b40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:20.091 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 >> 172.21.15.192:0/1549018724 conn(0x7f1ae80fae30 msgr2=0x7f1ae8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:42:20.091 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 shutdown_connections 2023-08-11T15:42:20.092 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:42:20.088+0000 7f1aede64700 1 -- 172.21.15.192:0/1549018724 wait complete. 2023-08-11T15:42:20.480 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.8.service 2023-08-11T15:42:20.482 INFO:tasks.cephadm:Deploying osd.9 on smithi032 with /dev/vg_nvme/lv_1... 2023-08-11T15:42:20.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-11T15:42:20.512 INFO:journalctl@ceph.osd.8.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:42:21.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:21.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:21.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:21.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:21.222 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:42:21.295 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.295 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.295 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:21.295 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:21.296 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.296 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:21.296 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.296 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.296 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:21.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:21.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:21.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:21.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:21.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:21.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:42:21 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:42:21.489+0000 7f5496729640 -1 osd.8 0 log_to_monitors true 2023-08-11T15:42:22.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:21 smithi032 ceph-mon[97103]: from='osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T15:42:22.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:21 smithi032 ceph-mon[97103]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T15:42:22.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:21 smithi192 ceph-mon[130806]: from='osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T15:42:22.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:21 smithi192 ceph-mon[130806]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T15:42:22.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:21 smithi134 ceph-mon[130447]: from='osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T15:42:22.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:21 smithi134 ceph-mon[130447]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T15:42:22.965 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:42:22.951+0000 7f548bda1700 -1 osd.8 0 waiting for initial osdmap 2023-08-11T15:42:22.965 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-11T15:42:22.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: osdmap e57: 9 total, 8 up, 9 in 2023-08-11T15:42:22.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:22.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:42:22.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:42:22.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:22.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:22.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:22.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:22.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:22.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:22.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:22.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:22.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:22.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.204 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-11T15:42:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-11T15:42:23.205 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-08-11T15:42:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-08-11T15:42:23.206 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0297506 s, 352 MB/s 2023-08-11T15:42:23.206 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-08-11T15:42:23.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-11T15:42:23.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: osdmap e57: 9 total, 8 up, 9 in 2023-08-11T15:42:23.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:23.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:42:23.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:42:23.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:23.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:23.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:23.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:23.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.226 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:23.226 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:23.226 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.241 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-11T15:42:23.242 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: osdmap e57: 9 total, 8 up, 9 in 2023-08-11T15:42:23.242 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:23.243 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:42:23.243 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:42:23.243 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:23.243 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.243 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.243 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:23.243 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:23.244 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:23.244 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.244 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:23.245 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:23.245 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:23.337 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:42:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:42:22.963+0000 7f5484b90700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:42:23.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi032:vg_nvme/lv_1 2023-08-11T15:42:24.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:23 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi192 2023-08-11T15:42:24.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:23 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi192 to 5722M 2023-08-11T15:42:24.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:23 smithi134 ceph-mon[130447]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:42:24.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:23 smithi134 ceph-mon[130447]: osdmap e58: 9 total, 8 up, 9 in 2023-08-11T15:42:24.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:23 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:24.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:23 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:24.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:23 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi192 2023-08-11T15:42:24.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:23 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi192 to 5722M 2023-08-11T15:42:24.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:23 smithi032 ceph-mon[97103]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:42:24.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:23 smithi032 ceph-mon[97103]: osdmap e58: 9 total, 8 up, 9 in 2023-08-11T15:42:24.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:24.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:24.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:23 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi192 2023-08-11T15:42:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:23 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi192 to 5722M 2023-08-11T15:42:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:23 smithi192 ceph-mon[130806]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:42:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:23 smithi192 ceph-mon[130806]: osdmap e58: 9 total, 8 up, 9 in 2023-08-11T15:42:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:23 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:23 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:24.443 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:42:25.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:24 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:42:25.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:24 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:42:25.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:24 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:25.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:24 smithi032 ceph-mon[97103]: osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] boot 2023-08-11T15:42:25.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:24 smithi032 ceph-mon[97103]: osdmap e59: 9 total, 9 up, 9 in 2023-08-11T15:42:25.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:24 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:25.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:24 smithi032 ceph-mon[97103]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:25.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:24 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:42:25.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:24 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:42:25.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:24 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:25.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:24 smithi134 ceph-mon[130447]: osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] boot 2023-08-11T15:42:25.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:24 smithi134 ceph-mon[130447]: osdmap e59: 9 total, 9 up, 9 in 2023-08-11T15:42:25.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:24 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:25.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:24 smithi134 ceph-mon[130447]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:25.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:24 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:42:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:24 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:42:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:24 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:24 smithi192 ceph-mon[130806]: osd.8 [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] boot 2023-08-11T15:42:25.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:24 smithi192 ceph-mon[130806]: osdmap e59: 9 total, 9 up, 9 in 2023-08-11T15:42:25.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:24 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:42:25.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:24 smithi192 ceph-mon[130806]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T15:42:26.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 -- 172.21.15.32:0/511958099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a41009b0 msgr2=0x7f79a4100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 --2- 172.21.15.32:0/511958099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a41009b0 0x7f79a4100dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79940099c0 tx=0x7f7994009cd0 comp rx=0 tx=0).stop 2023-08-11T15:42:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 -- 172.21.15.32:0/511958099 shutdown_connections 2023-08-11T15:42:26.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 --2- 172.21.15.32:0/511958099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a41009b0 0x7f79a4100dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:26.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 --2- 172.21.15.32:0/511958099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a4102550 0x7f79a4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:26.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 --2- 172.21.15.32:0/511958099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 0x7f79a4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:26.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 -- 172.21.15.32:0/511958099 >> 172.21.15.32:0/511958099 conn(0x7f79a40fbf10 msgr2=0x7f79a40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:42:26.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 -- 172.21.15.32:0/511958099 shutdown_connections 2023-08-11T15:42:26.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.039+0000 7f79a9279700 1 -- 172.21.15.32:0/511958099 wait complete. 2023-08-11T15:42:26.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.040+0000 7f79a9279700 1 Processor -- start 2023-08-11T15:42:26.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a9279700 1 -- start start 2023-08-11T15:42:26.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a9279700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a41009b0 0x7f79a41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:26.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a9279700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 0x7f79a4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:26.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a9279700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a4102550 0x7f79a419fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a9279700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41a0470 con 0x7f79a4102550 2023-08-11T15:42:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a9279700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41a05b0 con 0x7f79a4101bb0 2023-08-11T15:42:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a9279700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41a0740 con 0x7f79a41009b0 2023-08-11T15:42:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a27fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 0x7f79a4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a4102550 0x7f79a419fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a41009b0 0x7f79a41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a27fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 0x7f79a4198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55690/0 (socket says 172.21.15.32:55690) 2023-08-11T15:42:26.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.041+0000 7f79a27fc700 1 -- 172.21.15.32:0/1646641524 learned_addr learned my addr 172.21.15.32:0/1646641524 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:42:26.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a37fe700 1 -- 172.21.15.32:0/1646641524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a41009b0 msgr2=0x7f79a41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:26.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a37fe700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a41009b0 0x7f79a41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:26.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a37fe700 1 -- 172.21.15.32:0/1646641524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 msgr2=0x7f79a4198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:26.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a37fe700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 0x7f79a4198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:26.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a37fe700 1 -- 172.21.15.32:0/1646641524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79940095f0 con 0x7f79a4102550 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a27fc700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 0x7f79a4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a37fe700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a4102550 0x7f79a419fea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f799c007f40 tx=0x7f799c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f798bfff700 1 -- 172.21.15.32:0/1646641524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c009ea0 con 0x7f79a4102550 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a41a0a20 con 0x7f79a4102550 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f798bfff700 1 -- 172.21.15.32:0/1646641524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f799c00f040 con 0x7f79a4102550 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f798bfff700 1 -- 172.21.15.32:0/1646641524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c004350 con 0x7f79a4102550 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.042+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41a0f20 con 0x7f79a4102550 2023-08-11T15:42:26.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.044+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a410c200 con 0x7f79a4102550 2023-08-11T15:42:26.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.047+0000 7f798bfff700 1 -- 172.21.15.32:0/1646641524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f799c00db20 con 0x7f79a4102550 2023-08-11T15:42:26.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.048+0000 7f798bfff700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f798c09ffd0 0x7f798c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:26.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.049+0000 7f798bfff700 1 -- 172.21.15.32:0/1646641524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f799c0cab40 con 0x7f79a4102550 2023-08-11T15:42:26.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.049+0000 7f79a2ffd700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f798c09ffd0 0x7f798c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:26.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.049+0000 7f79a2ffd700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f798c09ffd0 0x7f798c0a2490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7994009990 tx=0x7f7994005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:42:26.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:25 smithi032 ceph-mon[97103]: osdmap e60: 9 total, 9 up, 9 in 2023-08-11T15:42:26.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.051+0000 7f798bfff700 1 -- 172.21.15.32:0/1646641524 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f799c0951c0 con 0x7f79a4102550 2023-08-11T15:42:26.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:25 smithi134 ceph-mon[130447]: osdmap e60: 9 total, 9 up, 9 in 2023-08-11T15:42:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:26.270+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f79a4108550 con 0x7f798c09ffd0 2023-08-11T15:42:26.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:25 smithi192 ceph-mon[130806]: osdmap e60: 9 total, 9 up, 9 in 2023-08-11T15:42:27.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:26 smithi032 ceph-mon[97103]: osdmap e61: 9 total, 9 up, 9 in 2023-08-11T15:42:27.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:26 smithi032 ceph-mon[97103]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:26 smithi032 ceph-mon[97103]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:27.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:26 smithi032 ceph-mon[97103]: osdmap e62: 9 total, 9 up, 9 in 2023-08-11T15:42:27.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:26 smithi192 ceph-mon[130806]: osdmap e61: 9 total, 9 up, 9 in 2023-08-11T15:42:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:26 smithi192 ceph-mon[130806]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:26 smithi192 ceph-mon[130806]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:26 smithi192 ceph-mon[130806]: osdmap e62: 9 total, 9 up, 9 in 2023-08-11T15:42:27.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:26 smithi134 ceph-mon[130447]: osdmap e61: 9 total, 9 up, 9 in 2023-08-11T15:42:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:26 smithi134 ceph-mon[130447]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:26 smithi134 ceph-mon[130447]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:26 smithi134 ceph-mon[130447]: osdmap e62: 9 total, 9 up, 9 in 2023-08-11T15:42:29.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:28 smithi032 ceph-mon[97103]: osdmap e63: 9 total, 9 up, 9 in 2023-08-11T15:42:29.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:28 smithi032 ceph-mon[97103]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:29.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:28 smithi192 ceph-mon[130806]: osdmap e63: 9 total, 9 up, 9 in 2023-08-11T15:42:29.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:28 smithi192 ceph-mon[130806]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:29.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:28 smithi134 ceph-mon[130447]: osdmap e63: 9 total, 9 up, 9 in 2023-08-11T15:42:29.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:28 smithi134 ceph-mon[130447]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:30.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:29 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3402004613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b7e6dc0-e7d9-4003-a922-2764ad27b7c3"}]: dispatch 2023-08-11T15:42:30.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:29 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3402004613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b7e6dc0-e7d9-4003-a922-2764ad27b7c3"}]': finished 2023-08-11T15:42:30.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:29 smithi032 ceph-mon[97103]: osdmap e64: 10 total, 9 up, 10 in 2023-08-11T15:42:30.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:29 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:30.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:29 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3402004613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b7e6dc0-e7d9-4003-a922-2764ad27b7c3"}]: dispatch 2023-08-11T15:42:30.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:29 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3402004613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b7e6dc0-e7d9-4003-a922-2764ad27b7c3"}]': finished 2023-08-11T15:42:30.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:29 smithi192 ceph-mon[130806]: osdmap e64: 10 total, 9 up, 10 in 2023-08-11T15:42:30.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:29 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:30.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:29 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3402004613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b7e6dc0-e7d9-4003-a922-2764ad27b7c3"}]: dispatch 2023-08-11T15:42:30.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:29 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3402004613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b7e6dc0-e7d9-4003-a922-2764ad27b7c3"}]': finished 2023-08-11T15:42:30.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:29 smithi134 ceph-mon[130447]: osdmap e64: 10 total, 9 up, 10 in 2023-08-11T15:42:30.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:29 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:31.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:30 smithi192 ceph-mon[130806]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-08-11T15:42:31.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:30 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1490209260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:31.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:30 smithi134 ceph-mon[130447]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-08-11T15:42:31.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:30 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1490209260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:31.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:30 smithi032 ceph-mon[97103]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-08-11T15:42:31.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:30 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1490209260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:33.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:32 smithi032 ceph-mon[97103]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:42:33.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:32 smithi192 ceph-mon[130806]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:42:33.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:32 smithi134 ceph-mon[130447]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:42:35.421 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:35 smithi032 ceph-mon[97103]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-08-11T15:42:35.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:35 smithi134 ceph-mon[130447]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-08-11T15:42:35.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:35 smithi192 ceph-mon[130806]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-08-11T15:42:37.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:37 smithi032 ceph-mon[97103]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:42:37.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:37 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-11T15:42:37.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:37 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:37.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:37 smithi134 ceph-mon[130447]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:42:37.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:37 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-11T15:42:37.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:37 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:37.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:37 smithi192 ceph-mon[130806]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:42:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:37 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-11T15:42:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:37 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:38.264 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:38 smithi032 ceph-mon[97103]: Deploying daemon osd.9 on smithi032 2023-08-11T15:42:38.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:38 smithi134 ceph-mon[130447]: Deploying daemon osd.9 on smithi032 2023-08-11T15:42:38.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:38 smithi192 ceph-mon[130806]: Deploying daemon osd.9 on smithi032 2023-08-11T15:42:39.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:39 smithi134 ceph-mon[130447]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:42:39.521 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:39 smithi032 ceph-mon[97103]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:42:39.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:39 smithi192 ceph-mon[130806]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:42:41.437 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:41 smithi032 ceph-mon[97103]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-08-11T15:42:41.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:41 smithi134 ceph-mon[130447]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-08-11T15:42:41.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:41 smithi192 ceph-mon[130806]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-08-11T15:42:43.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:42 smithi032 ceph-mon[97103]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:43.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:42 smithi192 ceph-mon[130806]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:43.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:42 smithi134 ceph-mon[130447]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:44.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:44 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:44.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:44 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:44.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:44 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:44.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:44 smithi032 ceph-mon[97103]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:44.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:44 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:44.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:44 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:44.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:44 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:44.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:44 smithi134 ceph-mon[130447]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:45.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:44 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:45.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:44 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:45.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:44 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:45.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:44 smithi192 ceph-mon[130806]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:47.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.131+0000 7f798bfff700 1 -- 172.21.15.32:0/1646641524 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f79a4108550 con 0x7f798c09ffd0 2023-08-11T15:42:47.136 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 9 on host 'smithi032' 2023-08-11T15:42:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f798c09ffd0 msgr2=0x7f798c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f798c09ffd0 0x7f798c0a2490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7994009990 tx=0x7f7994005fb0 comp rx=0 tx=0).stop 2023-08-11T15:42:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a4102550 msgr2=0x7f79a419fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a4102550 0x7f79a419fea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f799c007f40 tx=0x7f799c00d3b0 comp rx=0 tx=0).stop 2023-08-11T15:42:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 shutdown_connections 2023-08-11T15:42:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f798c09ffd0 0x7f798c0a2490 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79a4102550 0x7f79a419fea0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a41009b0 0x7f79a41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 --2- 172.21.15.32:0/1646641524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101bb0 0x7f79a4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.134+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 >> 172.21.15.32:0/1646641524 conn(0x7f79a40fbf10 msgr2=0x7f79a40fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:42:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.135+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 shutdown_connections 2023-08-11T15:42:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:42:47.135+0000 7f79a9279700 1 -- 172.21.15.32:0/1646641524 wait complete. 2023-08-11T15:42:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:47.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:47.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:47.512 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:47.512 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.512 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.512 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.512 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:47.570 DEBUG:teuthology.orchestra.run.smithi032:osd.9> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.9.service 2023-08-11T15:42:47.575 INFO:tasks.cephadm:Deploying osd.10 on smithi134 with /dev/vg_nvme/lv_1... 2023-08-11T15:42:47.575 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-11T15:42:47.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:47.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:42:47.665 INFO:journalctl@ceph.osd.9.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:42:47.803 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:42:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:42:47.689+0000 7f61691fc640 -1 osd.9 0 log_to_monitors true 2023-08-11T15:42:48.141 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:42:48.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:48 smithi134 ceph-mon[130447]: from='osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T15:42:48.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:48 smithi134 ceph-mon[130447]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T15:42:48.379 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:48 smithi032 ceph-mon[97103]: from='osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T15:42:48.379 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:48 smithi032 ceph-mon[97103]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T15:42:48.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:48 smithi192 ceph-mon[130806]: from='osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T15:42:48.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:48 smithi192 ceph-mon[130806]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T15:42:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:49 smithi134 ceph-mon[130447]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:49 smithi134 ceph-mon[130447]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-11T15:42:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:49 smithi134 ceph-mon[130447]: osdmap e65: 10 total, 9 up, 10 in 2023-08-11T15:42:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:49 smithi134 ceph-mon[130447]: from='osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:42:49.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:49 smithi134 ceph-mon[130447]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:42:49.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:42:49.114+0000 7f615e874700 -1 osd.9 0 waiting for initial osdmap 2023-08-11T15:42:49.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:42:49.134+0000 7f615a669700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:42:49.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-mon[97103]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:49.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-mon[97103]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-11T15:42:49.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-mon[97103]: osdmap e65: 10 total, 9 up, 10 in 2023-08-11T15:42:49.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:49.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-mon[97103]: from='osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:42:49.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:49 smithi032 ceph-mon[97103]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:42:49.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:49 smithi192 ceph-mon[130806]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:49.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:49 smithi192 ceph-mon[130806]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-11T15:42:49.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:49 smithi192 ceph-mon[130806]: osdmap e65: 10 total, 9 up, 10 in 2023-08-11T15:42:49.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:49 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:49.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:49 smithi192 ceph-mon[130806]: from='osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:42:49.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:49 smithi192 ceph-mon[130806]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-08-11T15:42:49.652 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-11T15:42:49.653 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-11T15:42:49.653 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-08-11T15:42:49.653 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-08-11T15:42:49.653 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111162 s, 943 MB/s 2023-08-11T15:42:49.654 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-08-11T15:42:50.089 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2023-08-11T15:42:50.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:50 smithi032 ceph-mon[97103]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:42:50.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:50 smithi032 ceph-mon[97103]: osdmap e66: 10 total, 9 up, 10 in 2023-08-11T15:42:50.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:50.264 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:50.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:50 smithi134 ceph-mon[130447]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:42:50.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:50 smithi134 ceph-mon[130447]: osdmap e66: 10 total, 9 up, 10 in 2023-08-11T15:42:50.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:50.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:50 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:50 smithi192 ceph-mon[130806]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-08-11T15:42:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:50 smithi192 ceph-mon[130806]: osdmap e66: 10 total, 9 up, 10 in 2023-08-11T15:42:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:50 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:50.643 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:42:51.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:42:51.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:42:51.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:51.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:51.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] boot 2023-08-11T15:42:51.178 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: osdmap e67: 10 total, 10 up, 10 in 2023-08-11T15:42:51.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:51.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.179 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:51.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:51.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:42:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:42:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] boot 2023-08-11T15:42:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: osdmap e67: 10 total, 10 up, 10 in 2023-08-11T15:42:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:42:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:42:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T15:42:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: osd.9 [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] boot 2023-08-11T15:42:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: osdmap e67: 10 total, 10 up, 10 in 2023-08-11T15:42:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:42:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:42:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:51.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:42:51.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:42:51.819 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 -- 172.21.15.134:0/3971802126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad4101440 msgr2=0x7ffad4103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:51.820 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 --2- 172.21.15.134:0/3971802126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad4101440 0x7ffad4103830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009990 tx=0x7ffabc009ca0 comp rx=0 tx=0).stop 2023-08-11T15:42:51.820 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 -- 172.21.15.134:0/3971802126 shutdown_connections 2023-08-11T15:42:51.820 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 --2- 172.21.15.134:0/3971802126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad41066a0 0x7ffad4108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:51.821 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 --2- 172.21.15.134:0/3971802126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad4101440 0x7ffad4103830 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:51.821 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 --2- 172.21.15.134:0/3971802126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad4103d70 0x7ffad4106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:51.821 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 -- 172.21.15.134:0/3971802126 >> 172.21.15.134:0/3971802126 conn(0x7ffad40fae10 msgr2=0x7ffad40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:42:51.821 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 -- 172.21.15.134:0/3971802126 shutdown_connections 2023-08-11T15:42:51.821 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.814+0000 7ffad9519700 1 -- 172.21.15.134:0/3971802126 wait complete. 2023-08-11T15:42:51.823 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 Processor -- start 2023-08-11T15:42:51.823 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 -- start start 2023-08-11T15:42:51.823 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad4101440 0x7ffad41965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:51.823 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4103d70 0x7ffad4196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:51.823 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad41066a0 0x7ffad419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:51.823 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad419e150 con 0x7ffad4101440 2023-08-11T15:42:51.824 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad419e290 con 0x7ffad41066a0 2023-08-11T15:42:51.824 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad9519700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad419e420 con 0x7ffad4103d70 2023-08-11T15:42:51.824 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad41066a0 0x7ffad419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:51.824 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad41066a0 0x7ffad419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53586/0 (socket says 172.21.15.134:53586) 2023-08-11T15:42:51.824 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad37fe700 1 -- 172.21.15.134:0/3912523445 learned_addr learned my addr 172.21.15.134:0/3912523445 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-08-11T15:42:51.824 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.815+0000 7ffad27fc700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4103d70 0x7ffad4196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad37fe700 1 -- 172.21.15.134:0/3912523445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4103d70 msgr2=0x7ffad4196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad37fe700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4103d70 0x7ffad4196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad37fe700 1 -- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad4101440 msgr2=0x7ffad41965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad2ffd700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad4101440 0x7ffad41965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad37fe700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad4101440 0x7ffad41965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad37fe700 1 -- 172.21.15.134:0/3912523445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0095f0 con 0x7ffad41066a0 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad2ffd700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad4101440 0x7ffad41965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:42:51.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad37fe700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad41066a0 0x7ffad419db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00ef90 tx=0x7ffacc00c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:42:51.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffacbfff700 1 -- 172.21.15.134:0/3912523445 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc018070 con 0x7ffad41066a0 2023-08-11T15:42:51.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffacbfff700 1 -- 172.21.15.134:0/3912523445 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffacc00f040 con 0x7ffad41066a0 2023-08-11T15:42:51.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffacbfff700 1 -- 172.21.15.134:0/3912523445 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc003ea0 con 0x7ffad41066a0 2023-08-11T15:42:51.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad419e6a0 con 0x7ffad41066a0 2023-08-11T15:42:51.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.816+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad419eba0 con 0x7ffad41066a0 2023-08-11T15:42:51.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.817+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad40fc990 con 0x7ffad41066a0 2023-08-11T15:42:51.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.818+0000 7ffacbfff700 1 -- 172.21.15.134:0/3912523445 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7ffacc00cb00 con 0x7ffad41066a0 2023-08-11T15:42:51.827 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.818+0000 7ffacbfff700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffac009ffd0 0x7ffac00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:42:51.828 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.819+0000 7ffacbfff700 1 -- 172.21.15.134:0/3912523445 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ffacc0ca650 con 0x7ffad41066a0 2023-08-11T15:42:51.828 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.819+0000 7ffad2ffd700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffac009ffd0 0x7ffac00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:42:51.828 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.819+0000 7ffad2ffd700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffac009ffd0 0x7ffac00a2490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b2e0 tx=0x7ffabc005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:42:51.828 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:51.823+0000 7ffacbfff700 1 -- 172.21.15.134:0/3912523445 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ffacc094c70 con 0x7ffad41066a0 2023-08-11T15:42:52.022 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:42:52.020+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ffad402ce30 con 0x7ffac009ffd0 2023-08-11T15:42:52.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:52 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi032 2023-08-11T15:42:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:52 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi032 to 4318M 2023-08-11T15:42:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:52 smithi134 ceph-mon[130447]: osdmap e68: 10 total, 10 up, 10 in 2023-08-11T15:42:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:52 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:52 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi032 2023-08-11T15:42:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:52 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi032 to 4318M 2023-08-11T15:42:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:52 smithi032 ceph-mon[97103]: osdmap e68: 10 total, 10 up, 10 in 2023-08-11T15:42:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:52 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:52 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi032 2023-08-11T15:42:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:52 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi032 to 4318M 2023-08-11T15:42:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:52 smithi192 ceph-mon[130806]: osdmap e68: 10 total, 10 up, 10 in 2023-08-11T15:42:52.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:42:52.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:42:52.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:52 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:42:53.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:53 smithi134 ceph-mon[130447]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:53 smithi134 ceph-mon[130447]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:53 smithi032 ceph-mon[97103]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:53 smithi032 ceph-mon[97103]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:53.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:53 smithi192 ceph-mon[130806]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:42:53.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:53 smithi192 ceph-mon[130806]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:55.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:55 smithi134 ceph-mon[130447]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:55.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:55 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/1637306547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]: dispatch 2023-08-11T15:42:55.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:55 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]: dispatch 2023-08-11T15:42:55.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:55 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]': finished 2023-08-11T15:42:55.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:55 smithi134 ceph-mon[130447]: osdmap e69: 11 total, 10 up, 11 in 2023-08-11T15:42:55.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:42:55.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:55 smithi134 ceph-mon[130447]: from='client.? 172.21.15.134:0/3824091297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:55 smithi032 ceph-mon[97103]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:55.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:55 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/1637306547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]: dispatch 2023-08-11T15:42:55.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:55 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]: dispatch 2023-08-11T15:42:55.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:55 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]': finished 2023-08-11T15:42:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:55 smithi032 ceph-mon[97103]: osdmap e69: 11 total, 10 up, 11 in 2023-08-11T15:42:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:42:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:55 smithi032 ceph-mon[97103]: from='client.? 172.21.15.134:0/3824091297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:55 smithi192 ceph-mon[130806]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:55 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/1637306547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]: dispatch 2023-08-11T15:42:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:55 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]: dispatch 2023-08-11T15:42:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:55 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8"}]': finished 2023-08-11T15:42:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:55 smithi192 ceph-mon[130806]: osdmap e69: 11 total, 10 up, 11 in 2023-08-11T15:42:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:42:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:55 smithi192 ceph-mon[130806]: from='client.? 172.21.15.134:0/3824091297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:42:57.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:57 smithi134 ceph-mon[130447]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:57 smithi032 ceph-mon[97103]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:57.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:57 smithi192 ceph-mon[130806]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:59.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:42:59 smithi134 ceph-mon[130447]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:59.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:42:59 smithi032 ceph-mon[97103]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:42:59.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:42:59 smithi192 ceph-mon[130806]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:01.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:01 smithi134 ceph-mon[130447]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:01.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:01 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-11T15:43:01.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:01 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:01 smithi032 ceph-mon[97103]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:01 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-11T15:43:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:01 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:01.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:01 smithi192 ceph-mon[130806]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:01.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:01 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-11T15:43:01.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:01 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:02.452 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:02 smithi134 ceph-mon[130447]: Deploying daemon osd.10 on smithi134 2023-08-11T15:43:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:02 smithi032 ceph-mon[97103]: Deploying daemon osd.10 on smithi134 2023-08-11T15:43:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:02 smithi192 ceph-mon[130806]: Deploying daemon osd.10 on smithi134 2023-08-11T15:43:03.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:03 smithi134 ceph-mon[130447]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:03.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:03 smithi032 ceph-mon[97103]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:03.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:03 smithi192 ceph-mon[130806]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:05 smithi134 ceph-mon[130447]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:05.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:05 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:05.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:05 smithi032 ceph-mon[97103]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:05.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:05.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:05.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:05 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:05.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:05 smithi192 ceph-mon[130806]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:05.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:05.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:05.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:05 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:07.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:07 smithi134 ceph-mon[130447]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:07.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:07 smithi032 ceph-mon[97103]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:07.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:07 smithi192 ceph-mon[130806]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:07.584 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 10 on host 'smithi134' 2023-08-11T15:43:07.585 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.582+0000 7ffacbfff700 1 -- 172.21.15.134:0/3912523445 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ffad402ce30 con 0x7ffac009ffd0 2023-08-11T15:43:07.586 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.584+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffac009ffd0 msgr2=0x7ffac00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:07.587 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.584+0000 7ffad9519700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffac009ffd0 0x7ffac00a2490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b2e0 tx=0x7ffabc005e80 comp rx=0 tx=0).stop 2023-08-11T15:43:07.587 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad41066a0 msgr2=0x7ffad419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:07.587 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad41066a0 0x7ffad419db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00ef90 tx=0x7ffacc00c450 comp rx=0 tx=0).stop 2023-08-11T15:43:07.588 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 shutdown_connections 2023-08-11T15:43:07.588 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ffac009ffd0 0x7ffac00a2490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:07.589 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad4101440 0x7ffad41965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:07.589 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffad41066a0 0x7ffad419db80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:07.589 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 --2- 172.21.15.134:0/3912523445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4103d70 0x7ffad4196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:07.589 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 >> 172.21.15.134:0/3912523445 conn(0x7ffad40fae10 msgr2=0x7ffad41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:07.590 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 shutdown_connections 2023-08-11T15:43:07.590 INFO:teuthology.orchestra.run.smithi134.stderr:2023-08-11T15:43:07.585+0000 7ffad9519700 1 -- 172.21.15.134:0/3912523445 wait complete. 2023-08-11T15:43:08.028 DEBUG:teuthology.orchestra.run.smithi134:osd.10> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.10.service 2023-08-11T15:43:08.030 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2023-08-11T15:43:08.030 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-11T15:43:08.060 INFO:journalctl@ceph.osd.10.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:43:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:43:08.645 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.646 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.646 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:08.646 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:08.646 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.646 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.647 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.647 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:08.647 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-mon[130447]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:08.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.804 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:08.804 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:08.804 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.804 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.805 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.805 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:08.805 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:08 smithi192 ceph-mon[130806]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:08.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:08.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:08.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:08.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:08.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:08 smithi032 ceph-mon[97103]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:08.971 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:43:08 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:43:08.887+0000 7fdcf2349640 -1 osd.10 0 log_to_monitors true 2023-08-11T15:43:09.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:09 smithi032 ceph-mon[97103]: from='osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T15:43:09.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:09 smithi032 ceph-mon[97103]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T15:43:09.820 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:09 smithi134 ceph-mon[130447]: from='osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T15:43:09.821 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:09 smithi134 ceph-mon[130447]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T15:43:09.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:09 smithi192 ceph-mon[130806]: from='osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T15:43:09.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:09 smithi192 ceph-mon[130806]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T15:43:09.940 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-11T15:43:09.941 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-11T15:43:09.941 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-08-11T15:43:09.941 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-08-11T15:43:09.942 INFO:teuthology.orchestra.run.smithi192.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113975 s, 920 MB/s 2023-08-11T15:43:09.942 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-08-11T15:43:10.236 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2023-08-11T15:43:10.790 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:43:10.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-11T15:43:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: osdmap e70: 11 total, 10 up, 11 in 2023-08-11T15:43:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:43:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:43:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi134 2023-08-11T15:43:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi134 to 4292M 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:10.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:10.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:10.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:10.971 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:43:10.555+0000 7fdce79c1700 -1 osd.10 0 waiting for initial osdmap 2023-08-11T15:43:10.971 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:43:10.566+0000 7fdce1fb3700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:43:10.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-11T15:43:10.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: osdmap e70: 11 total, 10 up, 11 in 2023-08-11T15:43:10.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:10.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:43:10.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:43:10.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:10.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi134 2023-08-11T15:43:10.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:10.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:10.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:10.976 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi134 to 4292M 2023-08-11T15:43:10.976 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:10.976 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:10.976 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:10.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-11T15:43:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: osdmap e70: 11 total, 10 up, 11 in 2023-08-11T15:43:11.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:11.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:43:11.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-08-11T15:43:11.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:11.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi134 2023-08-11T15:43:11.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:11.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:11.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:11.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:11.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:11.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:11.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi134 to 4292M 2023-08-11T15:43:11.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:11.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:11.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:11.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:11.686 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:11 smithi192 ceph-mon[130806]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:43:11.686 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:11 smithi192 ceph-mon[130806]: osdmap e71: 11 total, 10 up, 11 in 2023-08-11T15:43:11.687 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:11 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:11.688 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:11 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:11.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:11 smithi134 ceph-mon[130447]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:43:11.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:11 smithi134 ceph-mon[130447]: osdmap e71: 11 total, 10 up, 11 in 2023-08-11T15:43:11.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:11 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:11.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:11 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:12.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:11 smithi032 ceph-mon[97103]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-08-11T15:43:12.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:11 smithi032 ceph-mon[97103]: osdmap e71: 11 total, 10 up, 11 in 2023-08-11T15:43:12.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:11 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:12.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:11 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:12.067 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/3116367880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0100ab0 msgr2=0x7f3cb0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:12.067 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/3116367880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0100ab0 0x7f3cb0100f10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009990 tx=0x7f3ca0009ca0 comp rx=0 tx=0).stop 2023-08-11T15:43:12.067 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/3116367880 shutdown_connections 2023-08-11T15:43:12.067 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/3116367880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb0101450 0x7f3cb0107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:12.067 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/3116367880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb00ff8b0 0x7f3cb00ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:12.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/3116367880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0100ab0 0x7f3cb0100f10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:12.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/3116367880 >> 172.21.15.192:0/3116367880 conn(0x7f3cb00fae30 msgr2=0x7f3cb00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:12.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/3116367880 shutdown_connections 2023-08-11T15:43:12.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.065+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/3116367880 wait complete. 2023-08-11T15:43:12.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 Processor -- start 2023-08-11T15:43:12.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 -- start start 2023-08-11T15:43:12.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00ff8b0 0x7f3cb01955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:12.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb0100ab0 0x7f3cb0195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:12.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0101450 0x7f3cb019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:12.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb019d180 con 0x7f3cb00ff8b0 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb019d2c0 con 0x7f3cb0100ab0 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3cb4bcc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb019d450 con 0x7f3cb0101450 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3caed9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0101450 0x7f3cb019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3caed9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0101450 0x7f3cb019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57786/0 (socket says 172.21.15.192:57786) 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.066+0000 7f3caed9d700 1 -- 172.21.15.192:0/2850094932 learned_addr learned my addr 172.21.15.192:0/2850094932 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3cadd9b700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb0100ab0 0x7f3cb0195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3cae59c700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00ff8b0 0x7f3cb01955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:12.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3caed9d700 1 -- 172.21.15.192:0/2850094932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb0100ab0 msgr2=0x7f3cb0195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:12.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3caed9d700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb0100ab0 0x7f3cb0195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:12.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3caed9d700 1 -- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00ff8b0 msgr2=0x7f3cb01955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:12.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3caed9d700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00ff8b0 0x7f3cb01955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:12.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3caed9d700 1 -- 172.21.15.192:0/2850094932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00095f0 con 0x7f3cb0101450 2023-08-11T15:43:12.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3caed9d700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0101450 0x7f3cb019cbb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b9a0 tx=0x7f3ca800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:12.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3c9f7fe700 1 -- 172.21.15.192:0/2850094932 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800c930 con 0x7f3cb0101450 2023-08-11T15:43:12.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb019d730 con 0x7f3cb0101450 2023-08-11T15:43:12.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3c9f7fe700 1 -- 172.21.15.192:0/2850094932 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca8014440 con 0x7f3cb0101450 2023-08-11T15:43:12.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3c9f7fe700 1 -- 172.21.15.192:0/2850094932 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80075d0 con 0x7f3cb0101450 2023-08-11T15:43:12.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.067+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb019dc60 con 0x7f3cb0101450 2023-08-11T15:43:12.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.068+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb0106f00 con 0x7f3cb0101450 2023-08-11T15:43:12.074 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.069+0000 7f3c9f7fe700 1 -- 172.21.15.192:0/2850094932 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f3ca80145b0 con 0x7f3cb0101450 2023-08-11T15:43:12.074 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.070+0000 7f3c9f7fe700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c9809ff50 0x7f3c980a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:12.075 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.070+0000 7f3c9f7fe700 1 -- 172.21.15.192:0/2850094932 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f3ca8006e40 con 0x7f3cb0101450 2023-08-11T15:43:12.075 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.070+0000 7f3cae59c700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c9809ff50 0x7f3c980a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:12.075 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.071+0000 7f3cae59c700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c9809ff50 0x7f3c980a2410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4005950 tx=0x7f3ca40058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:12.075 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.073+0000 7f3c9f7fe700 1 -- 172.21.15.192:0/2850094932 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3ca8094b20 con 0x7f3cb0101450 2023-08-11T15:43:12.270 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:12.269+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cb002cda0 con 0x7f3c9809ff50 2023-08-11T15:43:12.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:43:12.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:43:12.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] boot 2023-08-11T15:43:12.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: osdmap e72: 11 total, 11 up, 11 in 2023-08-11T15:43:12.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:12.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: osdmap e73: 11 total, 11 up, 11 in 2023-08-11T15:43:12.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:12.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:43:12.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:43:12.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:43:12.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:12 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:12.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:43:12.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:43:12.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] boot 2023-08-11T15:43:12.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: osdmap e72: 11 total, 11 up, 11 in 2023-08-11T15:43:12.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:12.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: osdmap e73: 11 total, 11 up, 11 in 2023-08-11T15:43:12.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:12.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:43:12.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:43:12.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:43:12.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:12 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:13.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:43:13.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:43:13.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: osd.10 [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] boot 2023-08-11T15:43:13.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: osdmap e72: 11 total, 11 up, 11 in 2023-08-11T15:43:13.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:43:13.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: osdmap e73: 11 total, 11 up, 11 in 2023-08-11T15:43:13.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T15:43:13.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:43:13.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T15:43:13.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T15:43:13.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:12 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:14.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:13 smithi192 ceph-mon[130806]: osdmap e74: 11 total, 11 up, 11 in 2023-08-11T15:43:14.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:13 smithi134 ceph-mon[130447]: osdmap e74: 11 total, 11 up, 11 in 2023-08-11T15:43:14.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:13 smithi032 ceph-mon[97103]: osdmap e74: 11 total, 11 up, 11 in 2023-08-11T15:43:15.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:14 smithi192 ceph-mon[130806]: osdmap e75: 11 total, 11 up, 11 in 2023-08-11T15:43:15.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:14 smithi192 ceph-mon[130806]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:15.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:14 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/1420957665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]: dispatch 2023-08-11T15:43:15.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:14 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]: dispatch 2023-08-11T15:43:15.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:14 smithi192 ceph-mon[130806]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]': finished 2023-08-11T15:43:15.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:14 smithi192 ceph-mon[130806]: osdmap e76: 12 total, 11 up, 12 in 2023-08-11T15:43:15.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:14 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:14 smithi134 ceph-mon[130447]: osdmap e75: 11 total, 11 up, 11 in 2023-08-11T15:43:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:14 smithi134 ceph-mon[130447]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:14 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/1420957665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]: dispatch 2023-08-11T15:43:15.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:14 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]: dispatch 2023-08-11T15:43:15.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:14 smithi134 ceph-mon[130447]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]': finished 2023-08-11T15:43:15.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:14 smithi134 ceph-mon[130447]: osdmap e76: 12 total, 11 up, 12 in 2023-08-11T15:43:15.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:14 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:14 smithi032 ceph-mon[97103]: osdmap e75: 11 total, 11 up, 11 in 2023-08-11T15:43:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:14 smithi032 ceph-mon[97103]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:14 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/1420957665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]: dispatch 2023-08-11T15:43:15.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:14 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]: dispatch 2023-08-11T15:43:15.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:14 smithi032 ceph-mon[97103]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2799be08-a3bf-4a40-a968-0e091285d8c6"}]': finished 2023-08-11T15:43:15.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:14 smithi032 ceph-mon[97103]: osdmap e76: 12 total, 11 up, 12 in 2023-08-11T15:43:15.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:14 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:16.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:15 smithi192 ceph-mon[130806]: from='client.? 172.21.15.192:0/1116773931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:43:16.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:15 smithi134 ceph-mon[130447]: from='client.? 172.21.15.192:0/1116773931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:43:16.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:15 smithi032 ceph-mon[97103]: from='client.? 172.21.15.192:0/1116773931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T15:43:17.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:16 smithi134 ceph-mon[130447]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2023-08-11T15:43:17.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:16 smithi032 ceph-mon[97103]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2023-08-11T15:43:17.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:16 smithi192 ceph-mon[130806]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2023-08-11T15:43:19.392 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:19 smithi192 ceph-mon[130806]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:43:19.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:19 smithi134 ceph-mon[130447]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:43:19.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:19 smithi032 ceph-mon[97103]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T15:43:21.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:21 smithi192 ceph-mon[130806]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-08-11T15:43:21.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:21 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-11T15:43:21.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:21 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:21.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:21 smithi134 ceph-mon[130447]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-08-11T15:43:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:21 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-11T15:43:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:21 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:21.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:21 smithi032 ceph-mon[97103]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-08-11T15:43:21.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:21 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-11T15:43:21.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:21 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:22.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:22 smithi192 ceph-mon[130806]: Deploying daemon osd.11 on smithi192 2023-08-11T15:43:22.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:22 smithi134 ceph-mon[130447]: Deploying daemon osd.11 on smithi192 2023-08-11T15:43:22.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:22 smithi032 ceph-mon[97103]: Deploying daemon osd.11 on smithi192 2023-08-11T15:43:23.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:23 smithi192 ceph-mon[130806]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:43:23.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:23 smithi134 ceph-mon[130447]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:43:23.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:23 smithi032 ceph-mon[97103]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-08-11T15:43:25.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:25 smithi192 ceph-mon[130806]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:43:25.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:25 smithi134 ceph-mon[130447]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:43:25.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:25 smithi032 ceph-mon[97103]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T15:43:26.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:26.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:26.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:26 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:26.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:26.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:26.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:26 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:26 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:27.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:27 smithi192 ceph-mon[130806]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-08-11T15:43:27.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:27 smithi134 ceph-mon[130447]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-08-11T15:43:27.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:27 smithi032 ceph-mon[97103]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-08-11T15:43:27.785 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2023-08-11T15:43:27.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.783+0000 7f3c9f7fe700 1 -- 172.21.15.192:0/2850094932 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f3cb002cda0 con 0x7f3c9809ff50 2023-08-11T15:43:27.787 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c9809ff50 msgr2=0x7f3c980a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:27.787 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c9809ff50 0x7f3c980a2410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4005950 tx=0x7f3ca40058e0 comp rx=0 tx=0).stop 2023-08-11T15:43:27.787 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0101450 msgr2=0x7f3cb019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:27.787 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0101450 0x7f3cb019cbb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b9a0 tx=0x7f3ca800bcb0 comp rx=0 tx=0).stop 2023-08-11T15:43:27.788 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 shutdown_connections 2023-08-11T15:43:27.788 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c9809ff50 0x7f3c980a2410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:27.788 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00ff8b0 0x7f3cb01955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:27.788 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb0101450 0x7f3cb019cbb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:27.788 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 --2- 172.21.15.192:0/2850094932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb0100ab0 0x7f3cb0195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:27.788 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 >> 172.21.15.192:0/2850094932 conn(0x7f3cb00fae30 msgr2=0x7f3cb0104040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:27.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 shutdown_connections 2023-08-11T15:43:27.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:27.785+0000 7f3cb4bcc700 1 -- 172.21.15.192:0/2850094932 wait complete. 2023-08-11T15:43:28.172 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.11.service 2023-08-11T15:43:28.174 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-08-11T15:43:28.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd stat -f json 2023-08-11T15:43:28.206 INFO:journalctl@ceph.osd.11.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:43:28.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:28.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:28.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:28.911 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:28 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:28.911 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:28 smithi192 ceph-mon[130806]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:43:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:28 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:28 smithi032 ceph-mon[97103]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:29.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:28 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:29.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:28 smithi134 ceph-mon[130447]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:29.333 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:43:29 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:43:29.136+0000 7f83a09c1640 -1 osd.11 0 log_to_monitors true 2023-08-11T15:43:30.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:29 smithi192 ceph-mon[130806]: from='osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T15:43:30.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:29 smithi192 ceph-mon[130806]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T15:43:30.092 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:29 smithi032 ceph-mon[97103]: from='osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T15:43:30.092 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:29 smithi032 ceph-mon[97103]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T15:43:30.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:29 smithi134 ceph-mon[130447]: from='osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T15:43:30.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:29 smithi134 ceph-mon[130447]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T15:43:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/621415354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 msgr2=0x7fd8a4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/621415354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 0x7fd8a4108e00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b0d0 tx=0x7fd89c00b3e0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/621415354 shutdown_connections 2023-08-11T15:43:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/621415354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 0x7fd8a4108e00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/621415354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41009b0 0x7fd8a4100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/621415354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 0x7fd8a4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/621415354 >> 172.21.15.32:0/621415354 conn(0x7fd8a40fbf10 msgr2=0x7fd8a40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/621415354 shutdown_connections 2023-08-11T15:43:30.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.340+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/621415354 wait complete. 2023-08-11T15:43:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 Processor -- start 2023-08-11T15:43:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 -- start start 2023-08-11T15:43:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41009b0 0x7fd8a4196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 0x7fd8a4196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 0x7fd8a419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a419e320 con 0x7fd8a4102550 2023-08-11T15:43:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a419e460 con 0x7fd8a4101bb0 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.341+0000 7fd8a9ff8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a419e5f0 con 0x7fd8a41009b0 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 0x7fd8a4196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 0x7fd8a419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 0x7fd8a4196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50982/0 (socket says 172.21.15.32:50982) 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a2ffd700 1 -- 172.21.15.32:0/44034819 learned_addr learned my addr 172.21.15.32:0/44034819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a37fe700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41009b0 0x7fd8a4196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a3fff700 1 -- 172.21.15.32:0/44034819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41009b0 msgr2=0x7fd8a4196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a3fff700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41009b0 0x7fd8a4196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a3fff700 1 -- 172.21.15.32:0/44034819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 msgr2=0x7fd8a4196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a3fff700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 0x7fd8a4196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a3fff700 1 -- 172.21.15.32:0/44034819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c009d00 con 0x7fd8a4102550 2023-08-11T15:43:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a2ffd700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 0x7fd8a4196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:43:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.342+0000 7fd8a3fff700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 0x7fd8a419dd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009fd0 tx=0x7fd89c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.343+0000 7fd8a0ff9700 1 -- 172.21.15.32:0/44034819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00e040 con 0x7fd8a4102550 2023-08-11T15:43:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.343+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a419e870 con 0x7fd8a4102550 2023-08-11T15:43:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.343+0000 7fd8a0ff9700 1 -- 172.21.15.32:0/44034819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89c01dd50 con 0x7fd8a4102550 2023-08-11T15:43:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.343+0000 7fd8a0ff9700 1 -- 172.21.15.32:0/44034819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c01bd50 con 0x7fd8a4102550 2023-08-11T15:43:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.343+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a419ece0 con 0x7fd8a4102550 2023-08-11T15:43:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.344+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a4107ef0 con 0x7fd8a4102550 2023-08-11T15:43:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.346+0000 7fd8a0ff9700 1 -- 172.21.15.32:0/44034819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fd89c01b450 con 0x7fd8a4102550 2023-08-11T15:43:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.347+0000 7fd8a0ff9700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd88c09ff50 0x7fd88c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.347+0000 7fd8a0ff9700 1 -- 172.21.15.32:0/44034819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fd89c0ccff0 con 0x7fd8a4102550 2023-08-11T15:43:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.348+0000 7fd8a37fe700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd88c09ff50 0x7fd88c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.348+0000 7fd8a37fe700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd88c09ff50 0x7fd88c0a2410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4101a10 tx=0x7fd894007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.350+0000 7fd8a0ff9700 1 -- 172.21.15.32:0/44034819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd89c05fc30 con 0x7fd8a4102550 2023-08-11T15:43:30.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.570+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd8a410c210 con 0x7fd8a4102550 2023-08-11T15:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.570+0000 7fd8a0ff9700 1 -- 172.21.15.32:0/44034819 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7fd89c097380 con 0x7fd8a4102550 2023-08-11T15:43:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd88c09ff50 msgr2=0x7fd88c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd88c09ff50 0x7fd88c0a2410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4101a10 tx=0x7fd894007400 comp rx=0 tx=0).stop 2023-08-11T15:43:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 msgr2=0x7fd8a419dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:30.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 0x7fd8a419dd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009fd0 tx=0x7fd89c009440 comp rx=0 tx=0).stop 2023-08-11T15:43:30.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 shutdown_connections 2023-08-11T15:43:30.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd88c09ff50 0x7fd88c0a2410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a4102550 0x7fd8a419dd50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41009b0 0x7fd8a4196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 --2- 172.21.15.32:0/44034819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8a4101bb0 0x7fd8a4196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:30.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 >> 172.21.15.32:0/44034819 conn(0x7fd8a40fbf10 msgr2=0x7fd8a4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:30.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 shutdown_connections 2023-08-11T15:43:30.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:30.573+0000 7fd8a9ff8700 1 -- 172.21.15.32:0/44034819 wait complete. 2023-08-11T15:43:31.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-11T15:43:31.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: osdmap e77: 12 total, 11 up, 12 in 2023-08-11T15:43:31.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:31.006 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:43:31.006 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:43:31.006 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:31.006 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: Detected new or changed devices on smithi192 2023-08-11T15:43:31.006 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.007 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.008 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi192 to 4292M 2023-08-11T15:43:31.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:31.010 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:31.010 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.011 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:31.011 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:31.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:31.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:30 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/44034819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T15:43:31.014 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1691768591,"num_in_osds":12,"osd_in_since":1691768594,"num_remapped_pgs":0} 2023-08-11T15:43:31.083 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:43:30.820+0000 7f8396039700 -1 osd.11 0 waiting for initial osdmap 2023-08-11T15:43:31.084 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:43:30.834+0000 7f838fe2a700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T15:43:31.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-11T15:43:31.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: osdmap e77: 12 total, 11 up, 12 in 2023-08-11T15:43:31.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:31.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:43:31.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:43:31.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:31.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: Detected new or changed devices on smithi192 2023-08-11T15:43:31.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi192 to 4292M 2023-08-11T15:43:31.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:31.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:31.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:31.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:31.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:31.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:30 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/44034819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T15:43:31.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-11T15:43:31.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: osdmap e77: 12 total, 11 up, 12 in 2023-08-11T15:43:31.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:31.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:43:31.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-08-11T15:43:31.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:31.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: Detected new or changed devices on smithi192 2023-08-11T15:43:31.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:43:31.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi192 to 4292M 2023-08-11T15:43:31.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.224 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:31.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:31.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.225 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:31.226 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:31.226 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:31.226 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:31.226 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:30 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/44034819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T15:43:32.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd stat -f json 2023-08-11T15:43:32.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:31 smithi192 ceph-mon[130806]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:43:32.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:31 smithi192 ceph-mon[130806]: osdmap e78: 12 total, 11 up, 12 in 2023-08-11T15:43:32.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:32.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:31 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:32.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:31 smithi032 ceph-mon[97103]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:43:32.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:31 smithi032 ceph-mon[97103]: osdmap e78: 12 total, 11 up, 12 in 2023-08-11T15:43:32.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:32.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:31 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:32.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:31 smithi134 ceph-mon[130447]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-08-11T15:43:32.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:31 smithi134 ceph-mon[130447]: osdmap e78: 12 total, 11 up, 12 in 2023-08-11T15:43:32.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:32.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:31 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:32.761 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:43:33.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:32 smithi192 ceph-mon[130806]: purged_snaps scrub starts 2023-08-11T15:43:33.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:32 smithi192 ceph-mon[130806]: purged_snaps scrub ok 2023-08-11T15:43:33.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:32 smithi192 ceph-mon[130806]: osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] boot 2023-08-11T15:43:33.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:32 smithi192 ceph-mon[130806]: osdmap e79: 12 total, 12 up, 12 in 2023-08-11T15:43:33.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:32 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:33.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:32 smithi192 ceph-mon[130806]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:33.086 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:32 smithi032 ceph-mon[97103]: purged_snaps scrub starts 2023-08-11T15:43:33.086 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:32 smithi032 ceph-mon[97103]: purged_snaps scrub ok 2023-08-11T15:43:33.086 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:32 smithi032 ceph-mon[97103]: osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] boot 2023-08-11T15:43:33.087 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:32 smithi032 ceph-mon[97103]: osdmap e79: 12 total, 12 up, 12 in 2023-08-11T15:43:33.087 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:32 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:33.087 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:32 smithi032 ceph-mon[97103]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:33.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:32 smithi134 ceph-mon[130447]: purged_snaps scrub starts 2023-08-11T15:43:33.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:32 smithi134 ceph-mon[130447]: purged_snaps scrub ok 2023-08-11T15:43:33.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:32 smithi134 ceph-mon[130447]: osd.11 [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] boot 2023-08-11T15:43:33.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:32 smithi134 ceph-mon[130447]: osdmap e79: 12 total, 12 up, 12 in 2023-08-11T15:43:33.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:32 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:43:33.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:32 smithi134 ceph-mon[130447]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T15:43:33.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:33 smithi032 ceph-mon[97103]: osdmap e80: 12 total, 12 up, 12 in 2023-08-11T15:43:34.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:33 smithi192 ceph-mon[130806]: osdmap e80: 12 total, 12 up, 12 in 2023-08-11T15:43:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 -- 172.21.15.32:0/1967418312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c102550 msgr2=0x7f536c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 --2- 172.21.15.32:0/1967418312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c102550 0x7f536c108e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f536400b0d0 tx=0x7f536400b3e0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 -- 172.21.15.32:0/1967418312 shutdown_connections 2023-08-11T15:43:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 --2- 172.21.15.32:0/1967418312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c102550 0x7f536c108e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 --2- 172.21.15.32:0/1967418312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c1009b0 0x7f536c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 --2- 172.21.15.32:0/1967418312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f536c101bb0 0x7f536c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 -- 172.21.15.32:0/1967418312 >> 172.21.15.32:0/1967418312 conn(0x7f536c0fbf10 msgr2=0x7f536c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 -- 172.21.15.32:0/1967418312 shutdown_connections 2023-08-11T15:43:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.204+0000 7f53717a8700 1 -- 172.21.15.32:0/1967418312 wait complete. 2023-08-11T15:43:34.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.205+0000 7f53717a8700 1 Processor -- start 2023-08-11T15:43:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.205+0000 7f53717a8700 1 -- start start 2023-08-11T15:43:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f53717a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c1009b0 0x7f536c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f53717a8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c101bb0 0x7f536c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f53717a8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f536c102550 0x7f536c19ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f53717a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1a04f0 con 0x7f536c1009b0 2023-08-11T15:43:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f53717a8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1a0630 con 0x7f536c101bb0 2023-08-11T15:43:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f53717a8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1a07c0 con 0x7f536c102550 2023-08-11T15:43:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c1009b0 0x7f536c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c101bb0 0x7f536c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c101bb0 0x7f536c198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50998/0 (socket says 172.21.15.32:50998) 2023-08-11T15:43:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536a7fc700 1 -- 172.21.15.32:0/2142193279 learned_addr learned my addr 172.21.15.32:0/2142193279 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c1009b0 0x7f536c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52856/0 (socket says 172.21.15.32:52856) 2023-08-11T15:43:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536b7fe700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f536c102550 0x7f536c19ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536affd700 1 -- 172.21.15.32:0/2142193279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f536c102550 msgr2=0x7f536c19ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536affd700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f536c102550 0x7f536c19ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536affd700 1 -- 172.21.15.32:0/2142193279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c101bb0 msgr2=0x7f536c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536affd700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c101bb0 0x7f536c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536affd700 1 -- 172.21.15.32:0/2142193279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c009520 con 0x7f536c1009b0 2023-08-11T15:43:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.206+0000 7f536a7fc700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c101bb0 0x7f536c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:43:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.207+0000 7f536affd700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c1009b0 0x7f536c1989b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e9f0 tx=0x7f535c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.207+0000 7f5353fff700 1 -- 172.21.15.32:0/2142193279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c00cb70 con 0x7f536c1009b0 2023-08-11T15:43:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.207+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5364009d00 con 0x7f536c1009b0 2023-08-11T15:43:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.207+0000 7f5353fff700 1 -- 172.21.15.32:0/2142193279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f535c00ee90 con 0x7f536c1009b0 2023-08-11T15:43:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.207+0000 7f5353fff700 1 -- 172.21.15.32:0/2142193279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c018680 con 0x7f536c1009b0 2023-08-11T15:43:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.207+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c1a0e10 con 0x7f536c1009b0 2023-08-11T15:43:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.208+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c193000 con 0x7f536c1009b0 2023-08-11T15:43:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.210+0000 7f5353fff700 1 -- 172.21.15.32:0/2142193279 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f535c010430 con 0x7f536c1009b0 2023-08-11T15:43:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.211+0000 7f5353fff700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f535409ff50 0x7f53540a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.212+0000 7f5353fff700 1 -- 172.21.15.32:0/2142193279 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f535c014070 con 0x7f536c1009b0 2023-08-11T15:43:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.212+0000 7f536a7fc700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f535409ff50 0x7f53540a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.212+0000 7f536a7fc700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f535409ff50 0x7f53540a2410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5360011550 tx=0x7f5360009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.215+0000 7f5353fff700 1 -- 172.21.15.32:0/2142193279 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f535c095240 con 0x7f536c1009b0 2023-08-11T15:43:34.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:33 smithi134 ceph-mon[130447]: osdmap e80: 12 total, 12 up, 12 in 2023-08-11T15:43:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.424+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f536c10c210 con 0x7f536c1009b0 2023-08-11T15:43:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.424+0000 7f5353fff700 1 -- 172.21.15.32:0/2142193279 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f535c098860 con 0x7f536c1009b0 2023-08-11T15:43:34.426 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:34.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f535409ff50 msgr2=0x7f53540a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:34.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f535409ff50 0x7f53540a2410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5360011550 tx=0x7f5360009260 comp rx=0 tx=0).stop 2023-08-11T15:43:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c1009b0 msgr2=0x7f536c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c1009b0 0x7f536c1989b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e9f0 tx=0x7f535c00ed00 comp rx=0 tx=0).stop 2023-08-11T15:43:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 shutdown_connections 2023-08-11T15:43:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f535409ff50 0x7f53540a2410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f536c1009b0 0x7f536c1989b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f536c102550 0x7f536c19ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 --2- 172.21.15.32:0/2142193279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536c101bb0 0x7f536c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 >> 172.21.15.32:0/2142193279 conn(0x7f536c0fbf10 msgr2=0x7f536c107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 shutdown_connections 2023-08-11T15:43:34.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:34.427+0000 7f53717a8700 1 -- 172.21.15.32:0/2142193279 wait complete. 2023-08-11T15:43:34.846 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1691768611,"num_in_osds":12,"osd_in_since":1691768594,"num_remapped_pgs":0} 2023-08-11T15:43:34.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd dump --format=json 2023-08-11T15:43:35.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:34 smithi032 ceph-mon[97103]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:35.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:34 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2142193279' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T15:43:35.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:34 smithi134 ceph-mon[130447]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:35.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:34 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2142193279' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T15:43:35.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:34 smithi192 ceph-mon[130806]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:35.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:34 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2142193279' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T15:43:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:43:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.053+0000 7f70d8e4a700 1 -- 172.21.15.32:0/2532547983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d410d180 msgr2=0x7f70d410f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:37.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.053+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/2532547983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d410d180 0x7f70d410f5f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b0a0 tx=0x7f70cc00b3b0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.054+0000 7f70d8e4a700 1 -- 172.21.15.32:0/2532547983 shutdown_connections 2023-08-11T15:43:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.054+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/2532547983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d410d180 0x7f70d410f5f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.054+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/2532547983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d4072d50 0x7f70d4073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.054+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/2532547983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4073670 0x7f70d410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.054+0000 7f70d8e4a700 1 -- 172.21.15.32:0/2532547983 >> 172.21.15.32:0/2532547983 conn(0x7f70d40fc410 msgr2=0x7f70d40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.054+0000 7f70d8e4a700 1 -- 172.21.15.32:0/2532547983 shutdown_connections 2023-08-11T15:43:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.054+0000 7f70d8e4a700 1 -- 172.21.15.32:0/2532547983 wait complete. 2023-08-11T15:43:37.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 Processor -- start 2023-08-11T15:43:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 -- start start 2023-08-11T15:43:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4072d50 0x7f70d4198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d4073670 0x7f70d41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d410d180 0x7f70d419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419dd50 con 0x7f70d4073670 2023-08-11T15:43:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419dec0 con 0x7f70d410d180 2023-08-11T15:43:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.055+0000 7f70d8e4a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419e080 con 0x7f70d4072d50 2023-08-11T15:43:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d4073670 0x7f70d41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d4073670 0x7f70d41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52942/0 (socket says 172.21.15.32:52942) 2023-08-11T15:43:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 -- 172.21.15.32:0/242758670 learned_addr learned my addr 172.21.15.32:0/242758670 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70ca59c700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4072d50 0x7f70d4198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:37.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d2d9d700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d410d180 0x7f70d419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:37.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 -- 172.21.15.32:0/242758670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4072d50 msgr2=0x7f70d4198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:37.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4072d50 0x7f70d4198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 -- 172.21.15.32:0/242758670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d410d180 msgr2=0x7f70d419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d410d180 0x7f70d419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 -- 172.21.15.32:0/242758670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc009d00 con 0x7f70d4073670 2023-08-11T15:43:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.056+0000 7f70d259c700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d4073670 0x7f70d41992d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e9a0 tx=0x7f70bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.057+0000 7f70cbfff700 1 -- 172.21.15.32:0/242758670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc009790 con 0x7f70d4073670 2023-08-11T15:43:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.057+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d419e360 con 0x7f70d4073670 2023-08-11T15:43:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.057+0000 7f70cbfff700 1 -- 172.21.15.32:0/242758670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70bc00cbd0 con 0x7f70d4073670 2023-08-11T15:43:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.057+0000 7f70cbfff700 1 -- 172.21.15.32:0/242758670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc018680 con 0x7f70d4073670 2023-08-11T15:43:37.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.057+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d41a5bc0 con 0x7f70d4073670 2023-08-11T15:43:37.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.058+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d404e620 con 0x7f70d4073670 2023-08-11T15:43:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.060+0000 7f70cbfff700 1 -- 172.21.15.32:0/242758670 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f70bc00cd40 con 0x7f70d4073670 2023-08-11T15:43:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.061+0000 7f70cbfff700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f70c009ffd0 0x7f70c00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.061+0000 7f70cbfff700 1 -- 172.21.15.32:0/242758670 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70bc014070 con 0x7f70d4073670 2023-08-11T15:43:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.061+0000 7f70ca59c700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f70c009ffd0 0x7f70c00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.062+0000 7f70ca59c700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f70c009ffd0 0x7f70c00a2490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009c60 tx=0x7f70c4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.065+0000 7f70cbfff700 1 -- 172.21.15.32:0/242758670 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f70bc051c30 con 0x7f70d4073670 2023-08-11T15:43:37.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.276+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70d419a0f0 con 0x7f70d4073670 2023-08-11T15:43:37.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.277+0000 7f70cbfff700 1 -- 172.21.15.32:0/242758670 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18396 (secure 0 0 0) 0x7f70bc0951a0 con 0x7f70d4073670 2023-08-11T15:43:37.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:37.279 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:43:32.824336+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:43:37.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f70c009ffd0 msgr2=0x7f70c00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:37.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f70c009ffd0 0x7f70c00a2490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009c60 tx=0x7f70c4009260 comp rx=0 tx=0).stop 2023-08-11T15:43:37.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d4073670 msgr2=0x7f70d41992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:37.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d4073670 0x7f70d41992d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e9a0 tx=0x7f70bc00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 shutdown_connections 2023-08-11T15:43:37.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f70c009ffd0 0x7f70c00a2490 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70d410d180 0x7f70d419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4072d50 0x7f70d4198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 --2- 172.21.15.32:0/242758670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d4073670 0x7f70d41992d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:37.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 >> 172.21.15.32:0/242758670 conn(0x7f70d40fc410 msgr2=0x7f70d410f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:37.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 shutdown_connections 2023-08-11T15:43:37.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:37.280+0000 7f70d8e4a700 1 -- 172.21.15.32:0/242758670 wait complete. 2023-08-11T15:43:37.415 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:37 smithi032 ceph-mon[97103]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:37.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:37 smithi134 ceph-mon[130447]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:37 smithi192 ceph-mon[130806]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:37.865 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:43:37.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-08-11T15:43:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:38 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/242758670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:43:38.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:38 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/242758670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:43:38.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:38 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/242758670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:43:38.597 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:43:39.443 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:39 smithi032 ceph-mon[97103]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:39.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:39 smithi134 ceph-mon[130447]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:39.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:39 smithi192 ceph-mon[130806]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 -- 172.21.15.32:0/981724034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208106940 msgr2=0x7fb208106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 --2- 172.21.15.32:0/981724034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208106940 0x7fb208106d20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009990 tx=0x7fb1f8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 -- 172.21.15.32:0/981724034 shutdown_connections 2023-08-11T15:43:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 --2- 172.21.15.32:0/981724034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208106940 0x7fb208106d20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 --2- 172.21.15.32:0/981724034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb208101300 0x7fb20810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 --2- 172.21.15.32:0/981724034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 0x7fb208100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 -- 172.21.15.32:0/981724034 >> 172.21.15.32:0/981724034 conn(0x7fb2080fc410 msgr2=0x7fb2080fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 -- 172.21.15.32:0/981724034 shutdown_connections 2023-08-11T15:43:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.065+0000 7fb20e565700 1 -- 172.21.15.32:0/981724034 wait complete. 2023-08-11T15:43:40.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.066+0000 7fb20e565700 1 Processor -- start 2023-08-11T15:43:40.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.066+0000 7fb20e565700 1 -- start start 2023-08-11T15:43:40.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb20e565700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 0x7fb20819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:40.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb20e565700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208101300 0x7fb20819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:40.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb20e565700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb208106940 0x7fb208196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:40.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb20e565700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208196fd0 con 0x7fb208101300 2023-08-11T15:43:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb20e565700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208197140 con 0x7fb208106940 2023-08-11T15:43:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb20e565700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208197300 con 0x7fb208100940 2023-08-11T15:43:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb207fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 0x7fb20819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb207fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 0x7fb20819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:54052/0 (socket says 172.21.15.32:54052) 2023-08-11T15:43:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb207fff700 1 -- 172.21.15.32:0/3792326909 learned_addr learned my addr 172.21.15.32:0/3792326909 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb2077fe700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208101300 0x7fb20819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.067+0000 7fb20cb02700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb208106940 0x7fb208196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb207fff700 1 -- 172.21.15.32:0/3792326909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb208106940 msgr2=0x7fb208196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb207fff700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb208106940 0x7fb208196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb207fff700 1 -- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208101300 msgr2=0x7fb20819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb207fff700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208101300 0x7fb20819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb207fff700 1 -- 172.21.15.32:0/3792326909 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80095f0 con 0x7fb208100940 2023-08-11T15:43:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb2077fe700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208101300 0x7fb20819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:43:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb207fff700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 0x7fb20819c8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8005910 tx=0x7fb1f800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb2057fa700 1 -- 172.21.15.32:0/3792326909 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f801d070 con 0x7fb208100940 2023-08-11T15:43:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb208197580 con 0x7fb208100940 2023-08-11T15:43:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.068+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2081a38d0 con 0x7fb208100940 2023-08-11T15:43:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.069+0000 7fb2057fa700 1 -- 172.21.15.32:0/3792326909 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f8022470 con 0x7fb208100940 2023-08-11T15:43:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.069+0000 7fb2057fa700 1 -- 172.21.15.32:0/3792326909 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8017650 con 0x7fb208100940 2023-08-11T15:43:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.069+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f4005320 con 0x7fb208100940 2023-08-11T15:43:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.072+0000 7fb2057fa700 1 -- 172.21.15.32:0/3792326909 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fb1f800fb00 con 0x7fb208100940 2023-08-11T15:43:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.073+0000 7fb2057fa700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb1f009ff50 0x7fb1f00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.073+0000 7fb2057fa700 1 -- 172.21.15.32:0/3792326909 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb1f80cc3d0 con 0x7fb208100940 2023-08-11T15:43:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.073+0000 7fb2077fe700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb1f009ff50 0x7fb1f00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.074+0000 7fb2077fe700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb1f009ff50 0x7fb1f00a2410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb208106020 tx=0x7fb1fc009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.075+0000 7fb2057fa700 1 -- 172.21.15.32:0/3792326909 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb1f80965f0 con 0x7fb208100940 2023-08-11T15:43:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.282+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1f4005b50 con 0x7fb208100940 2023-08-11T15:43:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.283+0000 7fb2057fa700 1 -- 172.21.15.32:0/3792326909 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1f8099c10 con 0x7fb208100940 2023-08-11T15:43:40.284 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb1f009ff50 msgr2=0x7fb1f00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb1f009ff50 0x7fb1f00a2410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb208106020 tx=0x7fb1fc009190 comp rx=0 tx=0).stop 2023-08-11T15:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 msgr2=0x7fb20819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 0x7fb20819c8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8005910 tx=0x7fb1f800f690 comp rx=0 tx=0).stop 2023-08-11T15:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 shutdown_connections 2023-08-11T15:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb1f009ff50 0x7fb1f00a2410 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb208106940 0x7fb208196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb208100940 0x7fb20819c8e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 --2- 172.21.15.32:0/3792326909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb208101300 0x7fb20819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 >> 172.21.15.32:0/3792326909 conn(0x7fb2080fc410 msgr2=0x7fb2080fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 shutdown_connections 2023-08-11T15:43:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:40.285+0000 7fb20e565700 1 -- 172.21.15.32:0/3792326909 wait complete. 2023-08-11T15:43:40.676 INFO:tasks.cephadm:Adding mds.a on smithi032 2023-08-11T15:43:40.677 INFO:tasks.cephadm:Adding mds.d on smithi032 2023-08-11T15:43:40.677 INFO:tasks.cephadm:Adding mds.g on smithi032 2023-08-11T15:43:40.677 INFO:tasks.cephadm:Adding mds.j on smithi032 2023-08-11T15:43:40.677 INFO:tasks.cephadm:Adding mds.b on smithi134 2023-08-11T15:43:40.678 INFO:tasks.cephadm:Adding mds.e on smithi134 2023-08-11T15:43:40.678 INFO:tasks.cephadm:Adding mds.h on smithi134 2023-08-11T15:43:40.678 INFO:tasks.cephadm:Adding mds.k on smithi134 2023-08-11T15:43:40.678 INFO:tasks.cephadm:Adding mds.c on smithi192 2023-08-11T15:43:40.678 INFO:tasks.cephadm:Adding mds.f on smithi192 2023-08-11T15:43:40.679 INFO:tasks.cephadm:Adding mds.i on smithi192 2023-08-11T15:43:40.679 INFO:tasks.cephadm:Adding mds.l on smithi192 2023-08-11T15:43:40.679 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph orch apply mds all '12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2023-08-11T15:43:41.249 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:43:41.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:41 smithi192 ceph-mon[130806]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:41.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:41 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3792326909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:43:41.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:41 smithi134 ceph-mon[130447]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:41 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3792326909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:43:41.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:41 smithi032 ceph-mon[97103]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:41.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:41 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3792326909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:43:42.471 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 -- 172.21.15.192:0/107395028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c101440 msgr2=0x7fb12c103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:42.472 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 --2- 172.21.15.192:0/107395028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c101440 0x7fb12c103830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb120009990 tx=0x7fb120009ca0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.472 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 -- 172.21.15.192:0/107395028 shutdown_connections 2023-08-11T15:43:42.472 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 --2- 172.21.15.192:0/107395028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb12c1066a0 0x7fb12c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.472 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 --2- 172.21.15.192:0/107395028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c101440 0x7fb12c103830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.472 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 --2- 172.21.15.192:0/107395028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb12c103d70 0x7fb12c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.473 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 -- 172.21.15.192:0/107395028 >> 172.21.15.192:0/107395028 conn(0x7fb12c0fae30 msgr2=0x7fb12c0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:42.473 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 -- 172.21.15.192:0/107395028 shutdown_connections 2023-08-11T15:43:42.473 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.468+0000 7fb131b0c700 1 -- 172.21.15.192:0/107395028 wait complete. 2023-08-11T15:43:42.473 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 Processor -- start 2023-08-11T15:43:42.473 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 -- start start 2023-08-11T15:43:42.473 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb12c101440 0x7fb12c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:42.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c103d70 0x7fb12c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:42.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb12c1066a0 0x7fb12c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:42.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c1a0410 con 0x7fb12c101440 2023-08-11T15:43:42.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c1a0550 con 0x7fb12c1066a0 2023-08-11T15:43:42.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb131b0c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c1a06e0 con 0x7fb12c103d70 2023-08-11T15:43:42.474 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c103d70 0x7fb12c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:42.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c103d70 0x7fb12c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57910/0 (socket says 172.21.15.192:57910) 2023-08-11T15:43:42.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 -- 172.21.15.192:0/2481653816 learned_addr learned my addr 172.21.15.192:0/2481653816 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-08-11T15:43:42.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12bfff700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb12c1066a0 0x7fb12c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:42.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 -- 172.21.15.192:0/2481653816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb12c1066a0 msgr2=0x7fb12c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:42.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb12c1066a0 0x7fb12c19fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 -- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb12c101440 msgr2=0x7fb12c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:42.475 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb12c101440 0x7fb12c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.476 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.469+0000 7fb12affd700 1 -- 172.21.15.192:0/2481653816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1200095f0 con 0x7fb12c103d70 2023-08-11T15:43:42.476 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.470+0000 7fb12affd700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c103d70 0x7fb12c198dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00e9d0 tx=0x7fb11c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:42.476 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.470+0000 7fb128ff9700 1 -- 172.21.15.192:0/2481653816 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c009790 con 0x7fb12c103d70 2023-08-11T15:43:42.476 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.470+0000 7fb128ff9700 1 -- 172.21.15.192:0/2481653816 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb11c00cbd0 con 0x7fb12c103d70 2023-08-11T15:43:42.476 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.470+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12c1a09c0 con 0x7fb12c103d70 2023-08-11T15:43:42.476 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.470+0000 7fb128ff9700 1 -- 172.21.15.192:0/2481653816 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c018680 con 0x7fb12c103d70 2023-08-11T15:43:42.477 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.470+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c1a0f90 con 0x7fb12c103d70 2023-08-11T15:43:42.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.471+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c108260 con 0x7fb12c103d70 2023-08-11T15:43:42.478 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.472+0000 7fb128ff9700 1 -- 172.21.15.192:0/2481653816 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fb11c010430 con 0x7fb12c103d70 2023-08-11T15:43:42.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.472+0000 7fb128ff9700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb11409ffd0 0x7fb1140a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:42.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.473+0000 7fb128ff9700 1 -- 172.21.15.192:0/2481653816 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb11c014070 con 0x7fb12c103d70 2023-08-11T15:43:42.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.473+0000 7fb12b7fe700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb11409ffd0 0x7fb1140a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:42.479 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.473+0000 7fb12b7fe700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb11409ffd0 0x7fb1140a2490 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb1200095c0 tx=0x7fb120009550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:42.480 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.476+0000 7fb128ff9700 1 -- 172.21.15.192:0/2481653816 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb11c095280 con 0x7fb12c103d70 2023-08-11T15:43:42.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.671+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fb12c060d60 con 0x7fb11409ffd0 2023-08-11T15:43:42.681 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.680+0000 7fb128ff9700 1 -- 172.21.15.192:0/2481653816 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb12c060d60 con 0x7fb11409ffd0 2023-08-11T15:43:42.682 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2023-08-11T15:43:42.684 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb11409ffd0 msgr2=0x7fb1140a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:42.684 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb11409ffd0 0x7fb1140a2490 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb1200095c0 tx=0x7fb120009550 comp rx=0 tx=0).stop 2023-08-11T15:43:42.684 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c103d70 msgr2=0x7fb12c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:42.684 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c103d70 0x7fb12c198dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00e9d0 tx=0x7fb11c00ed90 comp rx=0 tx=0).stop 2023-08-11T15:43:42.685 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 shutdown_connections 2023-08-11T15:43:42.685 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb11409ffd0 0x7fb1140a2490 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.685 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb12c101440 0x7fb12c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.685 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb12c1066a0 0x7fb12c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.685 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 --2- 172.21.15.192:0/2481653816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb12c103d70 0x7fb12c198dc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:42.685 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 >> 172.21.15.192:0/2481653816 conn(0x7fb12c0fae30 msgr2=0x7fb12c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:42.685 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 shutdown_connections 2023-08-11T15:43:42.686 INFO:teuthology.orchestra.run.smithi192.stderr:2023-08-11T15:43:42.682+0000 7fb131b0c700 1 -- 172.21.15.192:0/2481653816 wait complete. 2023-08-11T15:43:43.023 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:43.023 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:43.024 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:43.024 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:43.024 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:43.025 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:43.025 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:43.025 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:43.025 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:42 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:43.114 DEBUG:teuthology.orchestra.run.smithi032:mds.a> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.a.service 2023-08-11T15:43:43.122 DEBUG:teuthology.orchestra.run.smithi032:mds.d> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.d.service 2023-08-11T15:43:43.167 DEBUG:teuthology.orchestra.run.smithi032:mds.g> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.g.service 2023-08-11T15:43:43.188 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:43.188 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:43.188 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:43.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:43.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:43.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:43.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:43.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:43.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:42 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:43.192 DEBUG:teuthology.orchestra.run.smithi032:mds.j> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.j.service 2023-08-11T15:43:43.206 INFO:journalctl@ceph.mds.a.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:43:43.209 DEBUG:teuthology.orchestra.run.smithi134:mds.b> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.b.service 2023-08-11T15:43:43.212 DEBUG:teuthology.orchestra.run.smithi134:mds.e> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.e.service 2023-08-11T15:43:43.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:43:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:43:43.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:43.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:43.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:43.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:42 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:43.223 DEBUG:teuthology.orchestra.run.smithi134:mds.h> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.h.service 2023-08-11T15:43:43.245 INFO:journalctl@ceph.mds.d.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:43:43.250 INFO:journalctl@ceph.mds.b.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:43:43.251 DEBUG:teuthology.orchestra.run.smithi134:mds.k> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.k.service 2023-08-11T15:43:43.252 INFO:journalctl@ceph.mds.e.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:43:43.273 INFO:journalctl@ceph.mds.g.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:43:43.279 INFO:journalctl@ceph.mds.h.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:43:43.280 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.c.service 2023-08-11T15:43:43.289 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.f.service 2023-08-11T15:43:43.290 INFO:journalctl@ceph.mds.j.smithi032.stdout:-- Logs begin at Fri 2023-08-11 15:18:36 UTC. -- 2023-08-11T15:43:43.307 INFO:journalctl@ceph.mds.k.smithi134.stdout:-- Logs begin at Fri 2023-08-11 15:14:31 UTC. -- 2023-08-11T15:43:43.314 INFO:journalctl@ceph.mds.c.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:43:43.315 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.i.service 2023-08-11T15:43:43.339 INFO:journalctl@ceph.mds.f.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:43:43.341 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.l.service 2023-08-11T15:43:43.366 INFO:journalctl@ceph.mds.i.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:43:43.367 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-08-11T15:43:43.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:43:43.397 INFO:journalctl@ceph.mds.l.smithi192.stdout:-- Logs begin at Fri 2023-08-11 15:14:45 UTC. -- 2023-08-11T15:43:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 -- 172.21.15.32:0/1802547625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c41013e0 msgr2=0x7fa0c41037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/1802547625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c41013e0 0x7fa0c41037d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009990 tx=0x7fa0b4009ca0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 -- 172.21.15.32:0/1802547625 shutdown_connections 2023-08-11T15:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/1802547625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c41013e0 0x7fa0c41037d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/1802547625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c4106640 0x7fa0c4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/1802547625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c4103d10 0x7fa0c4106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 -- 172.21.15.32:0/1802547625 >> 172.21.15.32:0/1802547625 conn(0x7fa0c40fad50 msgr2=0x7fa0c40fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 -- 172.21.15.32:0/1802547625 shutdown_connections 2023-08-11T15:43:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.661+0000 7fa0c97ff700 1 -- 172.21.15.32:0/1802547625 wait complete. 2023-08-11T15:43:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 Processor -- start 2023-08-11T15:43:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 -- start start 2023-08-11T15:43:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c41013e0 0x7fa0c41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c4103d10 0x7fa0c4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c4106640 0x7fa0c419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41a0360 con 0x7fa0c4106640 2023-08-11T15:43:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41a04a0 con 0x7fa0c41013e0 2023-08-11T15:43:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c97ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41a0630 con 0x7fa0c4103d10 2023-08-11T15:43:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.662+0000 7fa0c37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c4106640 0x7fa0c419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c41013e0 0x7fa0c41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c41013e0 0x7fa0c41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:48534/0 (socket says 172.21.15.32:48534) 2023-08-11T15:43:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 -- 172.21.15.32:0/867505472 learned_addr learned my addr 172.21.15.32:0/867505472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c27fc700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c4103d10 0x7fa0c4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 -- 172.21.15.32:0/867505472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c4103d10 msgr2=0x7fa0c4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:43.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c4103d10 0x7fa0c4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 -- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c4106640 msgr2=0x7fa0c419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c4106640 0x7fa0c419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 -- 172.21.15.32:0/867505472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40095f0 con 0x7fa0c41013e0 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c2ffd700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c41013e0 0x7fa0c41987e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4004740 tx=0x7fa0b4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0abfff700 1 -- 172.21.15.32:0/867505472 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b401d070 con 0x7fa0c41013e0 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.663+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c41a08b0 con 0x7fa0c41013e0 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.664+0000 7fa0abfff700 1 -- 172.21.15.32:0/867505472 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b4004830 con 0x7fa0c41013e0 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.664+0000 7fa0abfff700 1 -- 172.21.15.32:0/867505472 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40053c0 con 0x7fa0c41013e0 2023-08-11T15:43:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.664+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41a0cd0 con 0x7fa0c41013e0 2023-08-11T15:43:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.665+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c41081c0 con 0x7fa0c41013e0 2023-08-11T15:43:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.667+0000 7fa0abfff700 1 -- 172.21.15.32:0/867505472 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fa0b4005580 con 0x7fa0c41013e0 2023-08-11T15:43:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.668+0000 7fa0abfff700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa0ac09ffa0 0x7fa0ac0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.668+0000 7fa0abfff700 1 -- 172.21.15.32:0/867505472 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0b40cbff0 con 0x7fa0c41013e0 2023-08-11T15:43:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.668+0000 7fa0c27fc700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa0ac09ffa0 0x7fa0ac0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.669+0000 7fa0c27fc700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa0ac09ffa0 0x7fa0ac0a2460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009c60 tx=0x7fa0b8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.670+0000 7fa0abfff700 1 -- 172.21.15.32:0/867505472 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa0b4096230 con 0x7fa0c41013e0 2023-08-11T15:43:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.884+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0c410c130 con 0x7fa0c41013e0 2023-08-11T15:43:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.885+0000 7fa0abfff700 1 -- 172.21.15.32:0/867505472 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18396 (secure 0 0 0) 0x7fa0b4099850 con 0x7fa0c41013e0 2023-08-11T15:43:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:43.888 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:43:32.824336+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:43:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa0ac09ffa0 msgr2=0x7fa0ac0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa0ac09ffa0 0x7fa0ac0a2460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009c60 tx=0x7fa0b8009260 comp rx=0 tx=0).stop 2023-08-11T15:43:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c41013e0 msgr2=0x7fa0c41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c41013e0 0x7fa0c41987e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4004740 tx=0x7fa0b4009580 comp rx=0 tx=0).stop 2023-08-11T15:43:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 shutdown_connections 2023-08-11T15:43:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa0ac09ffa0 0x7fa0ac0a2460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c4106640 0x7fa0c419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c41013e0 0x7fa0c41987e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 --2- 172.21.15.32:0/867505472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c4103d10 0x7fa0c4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:43.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 >> 172.21.15.32:0/867505472 conn(0x7fa0c40fad50 msgr2=0x7fa0c40ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:43.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 shutdown_connections 2023-08-11T15:43:43.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:43.888+0000 7fa0c97ff700 1 -- 172.21.15.32:0/867505472 wait complete. 2023-08-11T15:43:43.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:43:43.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:43:43.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:43 smithi192 ceph-mon[130806]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:43:43.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:43 smithi192 ceph-mon[130806]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-08-11T15:43:43.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:43 smithi192 ceph-mon[130806]: Deploying daemon mds.l on smithi192 2023-08-11T15:43:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 -- 172.21.15.32:0/2218168268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5608100910 msgr2=0x7f5608100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 --2- 172.21.15.32:0/2218168268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5608100910 0x7f5608100d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56040099c0 tx=0x7f5604009cd0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 -- 172.21.15.32:0/2218168268 shutdown_connections 2023-08-11T15:43:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 --2- 172.21.15.32:0/2218168268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5608100910 0x7f5608100d30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 --2- 172.21.15.32:0/2218168268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56081024a0 0x7f5608108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 --2- 172.21.15.32:0/2218168268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608101b00 0x7f5608101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 -- 172.21.15.32:0/2218168268 >> 172.21.15.32:0/2218168268 conn(0x7f56080fbe60 msgr2=0x7f56080fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 -- 172.21.15.32:0/2218168268 shutdown_connections 2023-08-11T15:43:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.115+0000 7f560f3ff700 1 -- 172.21.15.32:0/2218168268 wait complete. 2023-08-11T15:43:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 Processor -- start 2023-08-11T15:43:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 -- start start 2023-08-11T15:43:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608100910 0x7f5608198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5608101b00 0x7f5608198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56081024a0 0x7f560819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56081a0380 con 0x7f56081024a0 2023-08-11T15:43:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56081a04c0 con 0x7f5608100910 2023-08-11T15:43:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.116+0000 7f560f3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56081a0650 con 0x7f5608101b00 2023-08-11T15:43:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608100910 0x7f5608198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608100910 0x7f5608198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:48556/0 (socket says 172.21.15.32:48556) 2023-08-11T15:43:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 -- 172.21.15.32:0/577427416 learned_addr learned my addr 172.21.15.32:0/577427416 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d99c700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56081024a0 0x7f560819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 -- 172.21.15.32:0/577427416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5608101b00 msgr2=0x7f5608198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5608101b00 0x7f5608198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 -- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56081024a0 msgr2=0x7f560819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56081024a0 0x7f560819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 -- 172.21.15.32:0/577427416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56040095f0 con 0x7f5608100910 2023-08-11T15:43:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d99c700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56081024a0 0x7f560819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:43:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.117+0000 7f560d19b700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608100910 0x7f5608198800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f56040098e0 tx=0x7f560400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.118+0000 7f55fe7fc700 1 -- 172.21.15.32:0/577427416 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560401d070 con 0x7f5608100910 2023-08-11T15:43:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.118+0000 7f55fe7fc700 1 -- 172.21.15.32:0/577427416 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5604005510 con 0x7f5608100910 2023-08-11T15:43:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.118+0000 7f55fe7fc700 1 -- 172.21.15.32:0/577427416 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604004c00 con 0x7f5608100910 2023-08-11T15:43:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.118+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56081a08d0 con 0x7f5608100910 2023-08-11T15:43:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.118+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56081a0ca0 con 0x7f5608100910 2023-08-11T15:43:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.119+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560810c170 con 0x7f5608100910 2023-08-11T15:43:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.121+0000 7f55fe7fc700 1 -- 172.21.15.32:0/577427416 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f560400fb80 con 0x7f5608100910 2023-08-11T15:43:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.122+0000 7f55fe7fc700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f55f409ff50 0x7f55f40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.122+0000 7f55fe7fc700 1 -- 172.21.15.32:0/577427416 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f56040cbca0 con 0x7f5608100910 2023-08-11T15:43:44.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.123+0000 7f560c99a700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f55f409ff50 0x7f55f40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.123+0000 7f560c99a700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f55f409ff50 0x7f55f40a2410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f55f8005950 tx=0x7f55f80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.124+0000 7f55fe7fc700 1 -- 172.21.15.32:0/577427416 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5604095ec0 con 0x7f5608100910 2023-08-11T15:43:44.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:43 smithi134 ceph-mon[130447]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:43:44.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:43 smithi134 ceph-mon[130447]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-08-11T15:43:44.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:43 smithi134 ceph-mon[130447]: Deploying daemon mds.l on smithi192 2023-08-11T15:43:44.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:43 smithi032 ceph-mon[97103]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:43:44.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:43 smithi032 ceph-mon[97103]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-08-11T15:43:44.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:43 smithi032 ceph-mon[97103]: Deploying daemon mds.l on smithi192 2023-08-11T15:43:44.333 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:43:44 smithi192 systemd[1]: Starting Ceph mds.l for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:43:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.334+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56081a0f80 con 0x7f5608100910 2023-08-11T15:43:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.335+0000 7f55fe7fc700 1 -- 172.21.15.32:0/577427416 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56040994e0 con 0x7f5608100910 2023-08-11T15:43:44.336 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:43:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f55f409ff50 msgr2=0x7f55f40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f55f409ff50 0x7f55f40a2410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f55f8005950 tx=0x7f55f80058e0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608100910 msgr2=0x7f5608198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608100910 0x7f5608198800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f56040098e0 tx=0x7f560400f690 comp rx=0 tx=0).stop 2023-08-11T15:43:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 shutdown_connections 2023-08-11T15:43:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f55f409ff50 0x7f55f40a2410 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56081024a0 0x7f560819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5608100910 0x7f5608198800 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 --2- 172.21.15.32:0/577427416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5608101b00 0x7f5608198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.337+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 >> 172.21.15.32:0/577427416 conn(0x7f56080fbe60 msgr2=0x7f56080fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.338+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 shutdown_connections 2023-08-11T15:43:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.338+0000 7f560f3ff700 1 -- 172.21.15.32:0/577427416 wait complete. 2023-08-11T15:43:44.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:43:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.561+0000 7f8e579ff700 1 -- 172.21.15.32:0/211691429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e50101b10 msgr2=0x7f8e50101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.561+0000 7f8e579ff700 1 --2- 172.21.15.32:0/211691429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e50101b10 0x7f8e50101f70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e400099c0 tx=0x7f8e40009cd0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.562+0000 7f8e579ff700 1 -- 172.21.15.32:0/211691429 shutdown_connections 2023-08-11T15:43:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.562+0000 7f8e579ff700 1 --2- 172.21.15.32:0/211691429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e501024b0 0x7f8e50108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.562+0000 7f8e579ff700 1 --2- 172.21.15.32:0/211691429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 0x7f8e50100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.562+0000 7f8e579ff700 1 --2- 172.21.15.32:0/211691429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e50101b10 0x7f8e50101f70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.562+0000 7f8e579ff700 1 -- 172.21.15.32:0/211691429 >> 172.21.15.32:0/211691429 conn(0x7f8e500fbe50 msgr2=0x7f8e500fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.562+0000 7f8e579ff700 1 -- 172.21.15.32:0/211691429 shutdown_connections 2023-08-11T15:43:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.562+0000 7f8e579ff700 1 -- 172.21.15.32:0/211691429 wait complete. 2023-08-11T15:43:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 Processor -- start 2023-08-11T15:43:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 -- start start 2023-08-11T15:43:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 0x7f8e50198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e50101b10 0x7f8e50198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e501024b0 0x7f8e5019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e501a03e0 con 0x7f8e501024b0 2023-08-11T15:43:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e501a0520 con 0x7f8e50101b10 2023-08-11T15:43:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e579ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e501a06b0 con 0x7f8e50100920 2023-08-11T15:43:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e5579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 0x7f8e50198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e5579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 0x7f8e50198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38754/0 (socket says 172.21.15.32:38754) 2023-08-11T15:43:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.563+0000 7f8e5579b700 1 -- 172.21.15.32:0/3993071435 learned_addr learned my addr 172.21.15.32:0/3993071435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e55f9c700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e501024b0 0x7f8e5019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e54f9a700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e50101b10 0x7f8e50198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e55f9c700 1 -- 172.21.15.32:0/3993071435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 msgr2=0x7f8e50198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e55f9c700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 0x7f8e50198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e55f9c700 1 -- 172.21.15.32:0/3993071435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e50101b10 msgr2=0x7f8e50198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e55f9c700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e50101b10 0x7f8e50198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e55f9c700 1 -- 172.21.15.32:0/3993071435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c009520 con 0x7f8e501024b0 2023-08-11T15:43:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e5579b700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 0x7f8e50198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:43:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e55f9c700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e501024b0 0x7f8e5019fe10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800efc0 tx=0x7f8e4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e467fc700 1 -- 172.21.15.32:0/3993071435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48018070 con 0x7f8e501024b0 2023-08-11T15:43:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e467fc700 1 -- 172.21.15.32:0/3993071435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4800f040 con 0x7f8e501024b0 2023-08-11T15:43:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e467fc700 1 -- 172.21.15.32:0/3993071435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800cd40 con 0x7f8e501024b0 2023-08-11T15:43:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.564+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e400095f0 con 0x7f8e501024b0 2023-08-11T15:43:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.565+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e501a0c60 con 0x7f8e501024b0 2023-08-11T15:43:44.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.566+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e50192f70 con 0x7f8e501024b0 2023-08-11T15:43:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.568+0000 7f8e467fc700 1 -- 172.21.15.32:0/3993071435 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f8e48003a00 con 0x7f8e501024b0 2023-08-11T15:43:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.569+0000 7f8e467fc700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8e3c09ffa0 0x7f8e3c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.569+0000 7f8e467fc700 1 -- 172.21.15.32:0/3993071435 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8e480ca9c0 con 0x7f8e501024b0 2023-08-11T15:43:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.569+0000 7f8e5579b700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8e3c09ffa0 0x7f8e3c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.570+0000 7f8e5579b700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8e3c09ffa0 0x7f8e3c0a2460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009fd0 tx=0x7f8e4c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.573+0000 7f8e467fc700 1 -- 172.21.15.32:0/3993071435 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8e48094be0 con 0x7f8e501024b0 2023-08-11T15:43:44.773 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:43:44 smithi192 podman[155720]: 2023-08-11T15:43:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.835+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e5010c180 con 0x7f8e501024b0 2023-08-11T15:43:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.837+0000 7f8e467fc700 1 -- 172.21.15.32:0/3993071435 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f8e48098200 con 0x7f8e501024b0 2023-08-11T15:43:44.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-11T15:43:44.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.839+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8e3c09ffa0 msgr2=0x7f8e3c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.839+0000 7f8e579ff700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8e3c09ffa0 0x7f8e3c0a2460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009fd0 tx=0x7f8e4c009260 comp rx=0 tx=0).stop 2023-08-11T15:43:44.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.839+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e501024b0 msgr2=0x7f8e5019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.839+0000 7f8e579ff700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e501024b0 0x7f8e5019fe10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800efc0 tx=0x7f8e4800c3a0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 shutdown_connections 2023-08-11T15:43:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8e3c09ffa0 0x7f8e3c0a2460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e501024b0 0x7f8e5019fe10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50100920 0x7f8e50198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 --2- 172.21.15.32:0/3993071435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e50101b10 0x7f8e50198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 >> 172.21.15.32:0/3993071435 conn(0x7f8e500fbe50 msgr2=0x7f8e50107400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 shutdown_connections 2023-08-11T15:43:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:44.840+0000 7f8e579ff700 1 -- 172.21.15.32:0/3993071435 wait complete. 2023-08-11T15:43:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2023-08-11T15:43:45.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/867505472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:43:45.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-mon[130806]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:45.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/577427416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:43:45.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.192:6832/408140602,v1:172.21.15.192:6833/408140602] up:boot 2023-08-11T15:43:45.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-mon[130806]: fsmap 1 up:standby 2023-08-11T15:43:45.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T15:43:45.088 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:43:44 smithi192 bash[155720]: f9d10d5e8d4b668256df83dd0fa409b383997c5d80e2e6538fb95e4b5c1a7ad3 2023-08-11T15:43:45.089 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-mds[155760]: -- 172.21.15.192:0/1602190465 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569ebc451e0 con 0x5569ebc1ac00 2023-08-11T15:43:45.089 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:43:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-l[155736]: starting mds.l at 2023-08-11T15:43:45.089 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:43:44 smithi192 systemd[1]: Started Ceph mds.l for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:43:45.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:44 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/867505472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:43:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:44 smithi134 ceph-mon[130447]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:44 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/577427416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:43:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:44 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.192:6832/408140602,v1:172.21.15.192:6833/408140602] up:boot 2023-08-11T15:43:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:44 smithi134 ceph-mon[130447]: fsmap 1 up:standby 2023-08-11T15:43:45.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:44 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T15:43:45.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:44 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/867505472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:43:45.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:44 smithi032 ceph-mon[97103]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:45.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:44 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/577427416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:43:45.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:44 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.192:6832/408140602,v1:172.21.15.192:6833/408140602] up:boot 2023-08-11T15:43:45.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:44 smithi032 ceph-mon[97103]: fsmap 1 up:standby 2023-08-11T15:43:45.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:44 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T15:43:46.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3993071435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:46.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:46.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:46.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:46.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:45 smithi134 ceph-mon[130447]: Deploying daemon mds.k on smithi134 2023-08-11T15:43:46.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3993071435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:46.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:46.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:45 smithi032 ceph-mon[97103]: Deploying daemon mds.k on smithi134 2023-08-11T15:43:46.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3993071435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:46.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:46.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:45 smithi192 ceph-mon[130806]: Deploying daemon mds.k on smithi134 2023-08-11T15:43:46.360 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:43:46 smithi134 systemd[1]: Starting Ceph mds.k for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:43:46.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:43:46.900 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:46 smithi134 ceph-mon[130447]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:46.901 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:43:46 smithi134 podman[154486]: 2023-08-11T15:43:47.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.084+0000 7f8d217ff700 1 -- 172.21.15.32:0/484619021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 msgr2=0x7f8d1c100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:47.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.084+0000 7f8d217ff700 1 --2- 172.21.15.32:0/484619021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c100d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0099c0 tx=0x7f8d0c009cd0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.085+0000 7f8d217ff700 1 -- 172.21.15.32:0/484619021 shutdown_connections 2023-08-11T15:43:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.085+0000 7f8d217ff700 1 --2- 172.21.15.32:0/484619021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c100d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.085+0000 7f8d217ff700 1 --2- 172.21.15.32:0/484619021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c1024b0 0x7f8d1c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.085+0000 7f8d217ff700 1 --2- 172.21.15.32:0/484619021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d1c101b10 0x7f8d1c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.085+0000 7f8d217ff700 1 -- 172.21.15.32:0/484619021 >> 172.21.15.32:0/484619021 conn(0x7f8d1c0fbe50 msgr2=0x7f8d1c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.085+0000 7f8d217ff700 1 -- 172.21.15.32:0/484619021 shutdown_connections 2023-08-11T15:43:47.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.085+0000 7f8d217ff700 1 -- 172.21.15.32:0/484619021 wait complete. 2023-08-11T15:43:47.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.086+0000 7f8d217ff700 1 Processor -- start 2023-08-11T15:43:47.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d217ff700 1 -- start start 2023-08-11T15:43:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d217ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d217ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d1c101b10 0x7f8d1c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d217ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c1024b0 0x7f8d1c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d217ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19e190 con 0x7f8d1c100920 2023-08-11T15:43:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d217ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19e2d0 con 0x7f8d1c1024b0 2023-08-11T15:43:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d217ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19e460 con 0x7f8d1c101b10 2023-08-11T15:43:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49294/0 (socket says 172.21.15.32:49294) 2023-08-11T15:43:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d1affd700 1 -- 172.21.15.32:0/2609856894 learned_addr learned my addr 172.21.15.32:0/2609856894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d1b7fe700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c1024b0 0x7f8d1c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.087+0000 7f8d1a7fc700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d1c101b10 0x7f8d1c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d1affd700 1 -- 172.21.15.32:0/2609856894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d1c101b10 msgr2=0x7f8d1c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d1affd700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d1c101b10 0x7f8d1c196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d1affd700 1 -- 172.21.15.32:0/2609856894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c1024b0 msgr2=0x7f8d1c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d1affd700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c1024b0 0x7f8d1c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d1affd700 1 -- 172.21.15.32:0/2609856894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0095f0 con 0x7f8d1c100920 2023-08-11T15:43:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d1affd700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c196610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009990 tx=0x7f8d0c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:47.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d03fff700 1 -- 172.21.15.32:0/2609856894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01d070 con 0x7f8d1c100920 2023-08-11T15:43:47.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c19e6e0 con 0x7f8d1c100920 2023-08-11T15:43:47.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d03fff700 1 -- 172.21.15.32:0/2609856894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d0c005030 con 0x7f8d1c100920 2023-08-11T15:43:47.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d03fff700 1 -- 172.21.15.32:0/2609856894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c004b30 con 0x7f8d1c100920 2023-08-11T15:43:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.088+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c19eb60 con 0x7f8d1c100920 2023-08-11T15:43:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.090+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c10c180 con 0x7f8d1c100920 2023-08-11T15:43:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.091+0000 7f8d03fff700 1 -- 172.21.15.32:0/2609856894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f8d0c004cb0 con 0x7f8d1c100920 2023-08-11T15:43:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.094+0000 7f8d03fff700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8d040a0020 0x7f8d040a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.094+0000 7f8d03fff700 1 -- 172.21.15.32:0/2609856894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d0c0cbb80 con 0x7f8d1c100920 2023-08-11T15:43:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.094+0000 7f8d1a7fc700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8d040a0020 0x7f8d040a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.095+0000 7f8d1a7fc700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8d040a0020 0x7f8d040a24e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005950 tx=0x7f8d1000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:47.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.098+0000 7f8d03fff700 1 -- 172.21.15.32:0/2609856894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8d0c095da0 con 0x7f8d1c100920 2023-08-11T15:43:47.220 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:43:46 smithi134 ceph-mds[154526]: -- 172.21.15.134:0/2232949113 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5576023ff1e0 con 0x5576023d4c00 2023-08-11T15:43:47.221 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:43:46 smithi134 bash[154486]: 3d38f3278566fd45cdda6ecf3faec3b0807358d54f5a0480ce6ac7b50bd9c87b 2023-08-11T15:43:47.221 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:43:46 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-k[154502]: starting mds.k at 2023-08-11T15:43:47.221 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:43:46 smithi134 systemd[1]: Started Ceph mds.k for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:43:47.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:46 smithi032 ceph-mon[97103]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:47.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:46 smithi192 ceph-mon[130806]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.364+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d1c19ee40 con 0x7f8d1c100920 2023-08-11T15:43:47.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.364+0000 7f8d03fff700 1 -- 172.21.15.32:0/2609856894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f8d1c19ee40 con 0x7f8d1c100920 2023-08-11T15:43:47.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:47.366 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-11T15:43:47.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8d040a0020 msgr2=0x7f8d040a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:47.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8d040a0020 0x7f8d040a24e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005950 tx=0x7f8d1000b410 comp rx=0 tx=0).stop 2023-08-11T15:43:47.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 msgr2=0x7f8d1c196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:47.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c196610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009990 tx=0x7f8d0c00f690 comp rx=0 tx=0).stop 2023-08-11T15:43:47.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 shutdown_connections 2023-08-11T15:43:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8d040a0020 0x7f8d040a24e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1c100920 0x7f8d1c196610 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c1024b0 0x7f8d1c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 --2- 172.21.15.32:0/2609856894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d1c101b10 0x7f8d1c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 >> 172.21.15.32:0/2609856894 conn(0x7f8d1c0fbe50 msgr2=0x7f8d1c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 shutdown_connections 2023-08-11T15:43:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:47.367+0000 7f8d217ff700 1 -- 172.21.15.32:0/2609856894 wait complete. 2023-08-11T15:43:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2023-08-11T15:43:48.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:48.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:48.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:48.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: Deploying daemon mds.j on smithi032 2023-08-11T15:43:48.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2609856894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:48.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6832/3616880880,v1:172.21.15.134:6833/3616880880] up:boot 2023-08-11T15:43:48.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: fsmap 2 up:standby 2023-08-11T15:43:48.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:47 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T15:43:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: Deploying daemon mds.j on smithi032 2023-08-11T15:43:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2609856894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:48.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6832/3616880880,v1:172.21.15.134:6833/3616880880] up:boot 2023-08-11T15:43:48.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: fsmap 2 up:standby 2023-08-11T15:43:48.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:47 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T15:43:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:48.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:48.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: Deploying daemon mds.j on smithi032 2023-08-11T15:43:48.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2609856894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:48.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6832/3616880880,v1:172.21.15.134:6833/3616880880] up:boot 2023-08-11T15:43:48.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: fsmap 2 up:standby 2023-08-11T15:43:48.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:47 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T15:43:49.123 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:48 smithi032 ceph-mon[97103]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:49.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:48 smithi192 ceph-mon[130806]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:49.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:43:49.459 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:43:49 smithi032 systemd[1]: Starting Ceph mds.j for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:43:49.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:48 smithi134 ceph-mon[130447]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 -- 172.21.15.32:0/129174300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 msgr2=0x7f89f4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 --2- 172.21.15.32:0/129174300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4100d50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f89f00099c0 tx=0x7f89f0009cd0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 -- 172.21.15.32:0/129174300 shutdown_connections 2023-08-11T15:43:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 --2- 172.21.15.32:0/129174300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4100d50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 --2- 172.21.15.32:0/129174300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f41024c0 0x7f89f4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 --2- 172.21.15.32:0/129174300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89f4101b20 0x7f89f4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 -- 172.21.15.32:0/129174300 >> 172.21.15.32:0/129174300 conn(0x7f89f40fbe60 msgr2=0x7f89f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 -- 172.21.15.32:0/129174300 shutdown_connections 2023-08-11T15:43:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.608+0000 7f89fbdff700 1 -- 172.21.15.32:0/129174300 wait complete. 2023-08-11T15:43:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.610+0000 7f89fbdff700 1 Processor -- start 2023-08-11T15:43:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.610+0000 7f89fbdff700 1 -- start start 2023-08-11T15:43:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.610+0000 7f89fbdff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89fbdff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89f4101b20 0x7f89f4198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89fbdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f41024c0 0x7f89f419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89fbdff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41a0390 con 0x7f89f4100930 2023-08-11T15:43:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89fbdff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41a04d0 con 0x7f89f4101b20 2023-08-11T15:43:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89fbdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41a0660 con 0x7f89f41024c0 2023-08-11T15:43:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49300/0 (socket says 172.21.15.32:49300) 2023-08-11T15:43:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 -- 172.21.15.32:0/79856215 learned_addr learned my addr 172.21.15.32:0/79856215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f939a700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89f4101b20 0x7f89f4198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89fa39c700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f41024c0 0x7f89f419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 -- 172.21.15.32:0/79856215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f41024c0 msgr2=0x7f89f419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f41024c0 0x7f89f419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 -- 172.21.15.32:0/79856215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89f4101b20 msgr2=0x7f89f4198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89f4101b20 0x7f89f4198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 -- 172.21.15.32:0/79856215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00095f0 con 0x7f89f4100930 2023-08-11T15:43:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.611+0000 7f89f9b9b700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4198810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fd0 tx=0x7f89f000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.612+0000 7f89eaffd700 1 -- 172.21.15.32:0/79856215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f001d070 con 0x7f89f4100930 2023-08-11T15:43:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.612+0000 7f89eaffd700 1 -- 172.21.15.32:0/79856215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89f0005050 con 0x7f89f4100930 2023-08-11T15:43:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.612+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f41a08e0 con 0x7f89f4100930 2023-08-11T15:43:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.612+0000 7f89eaffd700 1 -- 172.21.15.32:0/79856215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0004a70 con 0x7f89f4100930 2023-08-11T15:43:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.612+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f41a0d00 con 0x7f89f4100930 2023-08-11T15:43:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.613+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f410c190 con 0x7f89f4100930 2023-08-11T15:43:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.615+0000 7f89eaffd700 1 -- 172.21.15.32:0/79856215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f89f0005450 con 0x7f89f4100930 2023-08-11T15:43:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.616+0000 7f89eaffd700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89e009ffa0 0x7f89e00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.616+0000 7f89eaffd700 1 -- 172.21.15.32:0/79856215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89f00cbc10 con 0x7f89f4100930 2023-08-11T15:43:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.617+0000 7f89f939a700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89e009ffa0 0x7f89e00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.617+0000 7f89f939a700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89e009ffa0 0x7f89e00a2460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009c60 tx=0x7f89e4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.620+0000 7f89eaffd700 1 -- 172.21.15.32:0/79856215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f89f0095e30 con 0x7f89f4100930 2023-08-11T15:43:49.758 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:43:49 smithi032 podman[129756]: 2023-08-11T15:43:49.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.885+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89f410c3d0 con 0x7f89f4100930 2023-08-11T15:43:49.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.886+0000 7f89eaffd700 1 -- 172.21.15.32:0/79856215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f89f0099450 con 0x7f89f4100930 2023-08-11T15:43:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:49.888 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-11T15:43:49.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89e009ffa0 msgr2=0x7f89e00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:49.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89e009ffa0 0x7f89e00a2460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009c60 tx=0x7f89e4009260 comp rx=0 tx=0).stop 2023-08-11T15:43:49.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 msgr2=0x7f89f4198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:49.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4198810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fd0 tx=0x7f89f000f690 comp rx=0 tx=0).stop 2023-08-11T15:43:49.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 shutdown_connections 2023-08-11T15:43:49.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89e009ffa0 0x7f89e00a2460 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f4100930 0x7f89f4198810 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f41024c0 0x7f89f419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 --2- 172.21.15.32:0/79856215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89f4101b20 0x7f89f4198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:49.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.888+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 >> 172.21.15.32:0/79856215 conn(0x7f89f40fbe60 msgr2=0x7f89f41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:49.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.889+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 shutdown_connections 2023-08-11T15:43:49.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:49.889+0000 7f89fbdff700 1 -- 172.21.15.32:0/79856215 wait complete. 2023-08-11T15:43:49.892 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2023-08-11T15:43:50.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:49 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/79856215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:50.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:49 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:50.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:49 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:50.304 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:43:49 smithi032 bash[129756]: 10ecbc4db939fc6eedb6270ec72c770c40115adf672ed199d83b92db68ee165f 2023-08-11T15:43:50.304 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:43:49 smithi032 ceph-mds[129820]: -- 172.21.15.32:0/1454442912 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558659b131e0 con 0x558659ae8c00 2023-08-11T15:43:50.304 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:43:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-j[129796]: starting mds.j at 2023-08-11T15:43:50.304 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:43:49 smithi032 systemd[1]: Started Ceph mds.j for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:43:50.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:49 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/79856215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:50.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:49 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:50.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:49 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:50.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:49 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/79856215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:50.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:50.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:49 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:51.226 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:50 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6834/3298780897,v1:172.21.15.32:6835/3298780897] up:boot 2023-08-11T15:43:51.226 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:50 smithi032 ceph-mon[97103]: fsmap 3 up:standby 2023-08-11T15:43:51.226 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:50 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T15:43:51.226 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:51.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:51.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:51.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:51 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:51.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:51 smithi032 ceph-mon[97103]: Deploying daemon mds.d on smithi032 2023-08-11T15:43:51.227 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:51 smithi032 ceph-mon[97103]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6834/3298780897,v1:172.21.15.32:6835/3298780897] up:boot 2023-08-11T15:43:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: fsmap 3 up:standby 2023-08-11T15:43:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T15:43:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:51.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:51.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:51.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: Deploying daemon mds.d on smithi032 2023-08-11T15:43:51.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:51 smithi192 ceph-mon[130806]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6834/3298780897,v1:172.21.15.32:6835/3298780897] up:boot 2023-08-11T15:43:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: fsmap 3 up:standby 2023-08-11T15:43:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T15:43:51.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:51.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:51.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:51.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:51.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: Deploying daemon mds.d on smithi032 2023-08-11T15:43:51.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:51 smithi134 ceph-mon[130447]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:51.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:43:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 -- 172.21.15.32:0/3965496213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780101b20 msgr2=0x7f0780101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 --2- 172.21.15.32:0/3965496213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780101b20 0x7f0780101f80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07740099c0 tx=0x7f0774009cd0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 -- 172.21.15.32:0/3965496213 shutdown_connections 2023-08-11T15:43:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 --2- 172.21.15.32:0/3965496213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07801024c0 0x7f0780108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 --2- 172.21.15.32:0/3965496213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0780100930 0x7f0780100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 --2- 172.21.15.32:0/3965496213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780101b20 0x7f0780101f80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 -- 172.21.15.32:0/3965496213 >> 172.21.15.32:0/3965496213 conn(0x7f07800fbe60 msgr2=0x7f07800fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 -- 172.21.15.32:0/3965496213 shutdown_connections 2023-08-11T15:43:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.303+0000 7f0785fff700 1 -- 172.21.15.32:0/3965496213 wait complete. 2023-08-11T15:43:52.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 Processor -- start 2023-08-11T15:43:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 -- start start 2023-08-11T15:43:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780100930 0x7f07800746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0780101b20 0x7f0780072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07801024c0 0x7f0780073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800739f0 con 0x7f0780100930 2023-08-11T15:43:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780073b60 con 0x7f0780101b20 2023-08-11T15:43:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.305+0000 7f0785fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07801a06d0 con 0x7f07801024c0 2023-08-11T15:43:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780100930 0x7f07800746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780100930 0x7f07800746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49332/0 (socket says 172.21.15.32:49332) 2023-08-11T15:43:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 -- 172.21.15.32:0/902443805 learned_addr learned my addr 172.21.15.32:0/902443805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077effd700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0780101b20 0x7f0780072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077ffff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07801024c0 0x7f0780073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 -- 172.21.15.32:0/902443805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07801024c0 msgr2=0x7f0780073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07801024c0 0x7f0780073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 -- 172.21.15.32:0/902443805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0780101b20 msgr2=0x7f0780072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0780101b20 0x7f0780072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 -- 172.21.15.32:0/902443805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07740095f0 con 0x7f0780100930 2023-08-11T15:43:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077ffff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07801024c0 0x7f0780073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:43:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.306+0000 7f077f7fe700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780100930 0x7f07800746d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f077000e970 tx=0x7f077000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.307+0000 7f077cff9700 1 -- 172.21.15.32:0/902443805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770009790 con 0x7f0780100930 2023-08-11T15:43:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.307+0000 7f077cff9700 1 -- 172.21.15.32:0/902443805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f077000eec0 con 0x7f0780100930 2023-08-11T15:43:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.307+0000 7f077cff9700 1 -- 172.21.15.32:0/902443805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770003cf0 con 0x7f0780100930 2023-08-11T15:43:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.307+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07801a09b0 con 0x7f0780100930 2023-08-11T15:43:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.307+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07801a0e70 con 0x7f0780100930 2023-08-11T15:43:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.308+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0780107ed0 con 0x7f0780100930 2023-08-11T15:43:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.311+0000 7f077cff9700 1 -- 172.21.15.32:0/902443805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f077000ccd0 con 0x7f0780100930 2023-08-11T15:43:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.312+0000 7f077cff9700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f076809ffa0 0x7f07680a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.312+0000 7f077cff9700 1 -- 172.21.15.32:0/902443805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0770014070 con 0x7f0780100930 2023-08-11T15:43:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.312+0000 7f077effd700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f076809ffa0 0x7f07680a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.313+0000 7f077effd700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f076809ffa0 0x7f07680a2460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f077400b580 tx=0x7f0774005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.316+0000 7f077cff9700 1 -- 172.21.15.32:0/902443805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0770095210 con 0x7f0780100930 2023-08-11T15:43:52.553 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:43:52 smithi032 systemd[1]: Starting Ceph mds.d for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:43:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.589+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f078010c190 con 0x7f0780100930 2023-08-11T15:43:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.589+0000 7f077cff9700 1 -- 172.21.15.32:0/902443805 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f0770098830 con 0x7f0780100930 2023-08-11T15:43:52.591 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:52.591 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-11T15:43:52.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f076809ffa0 msgr2=0x7f07680a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:52.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f076809ffa0 0x7f07680a2460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f077400b580 tx=0x7f0774005fb0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780100930 msgr2=0x7f07800746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780100930 0x7f07800746d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f077000e970 tx=0x7f077000ec80 comp rx=0 tx=0).stop 2023-08-11T15:43:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 shutdown_connections 2023-08-11T15:43:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f076809ffa0 0x7f07680a2460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0780100930 0x7f07800746d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07801024c0 0x7f0780073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 --2- 172.21.15.32:0/902443805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0780101b20 0x7f0780072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 >> 172.21.15.32:0/902443805 conn(0x7f07800fbe60 msgr2=0x7f0780105010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 shutdown_connections 2023-08-11T15:43:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:52.592+0000 7f0785fff700 1 -- 172.21.15.32:0/902443805 wait complete. 2023-08-11T15:43:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 4 2023-08-11T15:43:52.910 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:43:52 smithi032 podman[130163]: 2023-08-11T15:43:52.911 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:52 smithi032 ceph-mon[97103]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:52.911 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:52 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/902443805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:53.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:52 smithi192 ceph-mon[130806]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:53.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:52 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/902443805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:53.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:52 smithi134 ceph-mon[130447]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:53.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:52 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/902443805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:53.303 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:43:52 smithi032 bash[130163]: 3c3d60154cd99cf0622f784c9ce0f40ed1501738dc7508faa153408ac610b4b5 2023-08-11T15:43:53.303 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:43:52 smithi032 ceph-mds[130205]: -- 172.21.15.32:0/1044696728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5611f77731e0 con 0x5611f7749000 2023-08-11T15:43:53.303 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:43:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-d[130181]: starting mds.d at 2023-08-11T15:43:53.303 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:43:52 smithi032 systemd[1]: Started Ceph mds.d for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:43:54.282 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.283 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.283 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.283 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:54.283 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:54.283 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:54.283 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: Deploying daemon mds.e on smithi134 2023-08-11T15:43:54.284 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6836/1824615419,v1:172.21.15.32:6837/1824615419] up:boot 2023-08-11T15:43:54.284 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: fsmap 4 up:standby 2023-08-11T15:43:54.284 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T15:43:54.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:54.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:54.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:54.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: Deploying daemon mds.e on smithi134 2023-08-11T15:43:54.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6836/1824615419,v1:172.21.15.32:6837/1824615419] up:boot 2023-08-11T15:43:54.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: fsmap 4 up:standby 2023-08-11T15:43:54.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T15:43:54.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:54.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:54.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:54.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:54.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: Deploying daemon mds.e on smithi134 2023-08-11T15:43:54.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6836/1824615419,v1:172.21.15.32:6837/1824615419] up:boot 2023-08-11T15:43:54.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: fsmap 4 up:standby 2023-08-11T15:43:54.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T15:43:54.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:43:54.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 -- 172.21.15.32:0/1147027776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac441024b0 msgr2=0x7fac44108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:54.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/1147027776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac441024b0 0x7fac44108d60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00b0d0 tx=0x7fac3c00b3e0 comp rx=0 tx=0).stop 2023-08-11T15:43:54.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 -- 172.21.15.32:0/1147027776 shutdown_connections 2023-08-11T15:43:54.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/1147027776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac44100920 0x7fac44100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:54.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/1147027776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac441024b0 0x7fac44108d60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:54.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/1147027776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac44101b10 0x7fac44101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:54.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 -- 172.21.15.32:0/1147027776 >> 172.21.15.32:0/1147027776 conn(0x7fac440fbe50 msgr2=0x7fac440fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:54.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 -- 172.21.15.32:0/1147027776 shutdown_connections 2023-08-11T15:43:54.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.831+0000 7fac4c1ff700 1 -- 172.21.15.32:0/1147027776 wait complete. 2023-08-11T15:43:54.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 Processor -- start 2023-08-11T15:43:54.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 -- start start 2023-08-11T15:43:54.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac44100920 0x7fac441988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:54.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac44101b10 0x7fac44198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:54.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac441024b0 0x7fac4419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:54.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac441a0450 con 0x7fac441024b0 2023-08-11T15:43:54.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac441a0590 con 0x7fac44100920 2023-08-11T15:43:54.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.832+0000 7fac4c1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac441a0720 con 0x7fac44101b10 2023-08-11T15:43:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac44100920 0x7fac441988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac4a79c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac441024b0 0x7fac4419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac44100920 0x7fac441988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:46464/0 (socket says 172.21.15.32:46464) 2023-08-11T15:43:54.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 -- 172.21.15.32:0/3688304180 learned_addr learned my addr 172.21.15.32:0/3688304180 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:54.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac4979a700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac44101b10 0x7fac44198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:54.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 -- 172.21.15.32:0/3688304180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac44101b10 msgr2=0x7fac44198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:54.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac44101b10 0x7fac44198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:54.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 -- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac441024b0 msgr2=0x7fac4419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:54.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac441024b0 0x7fac4419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:54.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 -- 172.21.15.32:0/3688304180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c009d00 con 0x7fac44100920 2023-08-11T15:43:54.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac4a79c700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac441024b0 0x7fac4419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:43:54.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac49f9b700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac44100920 0x7fac441988d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac4000e970 tx=0x7fac4000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:54.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac3affd700 1 -- 172.21.15.32:0/3688304180 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40009790 con 0x7fac44100920 2023-08-11T15:43:54.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.833+0000 7fac3affd700 1 -- 172.21.15.32:0/3688304180 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac4000c7e0 con 0x7fac44100920 2023-08-11T15:43:54.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.834+0000 7fac3affd700 1 -- 172.21.15.32:0/3688304180 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40003c70 con 0x7fac44100920 2023-08-11T15:43:54.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.834+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac441a0a00 con 0x7fac44100920 2023-08-11T15:43:54.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.834+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac441a0ec0 con 0x7fac44100920 2023-08-11T15:43:54.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.835+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4410c190 con 0x7fac44100920 2023-08-11T15:43:54.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.837+0000 7fac3affd700 1 -- 172.21.15.32:0/3688304180 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fac4000cc50 con 0x7fac44100920 2023-08-11T15:43:54.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.838+0000 7fac3affd700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fac3009ffa0 0x7fac300a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:54.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.838+0000 7fac3affd700 1 -- 172.21.15.32:0/3688304180 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fac40014070 con 0x7fac44100920 2023-08-11T15:43:54.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.838+0000 7fac4979a700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fac3009ffa0 0x7fac300a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:54.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.839+0000 7fac4979a700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fac3009ffa0 0x7fac300a2460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac34007be0 tx=0x7fac340073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:54.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:54.840+0000 7fac3affd700 1 -- 172.21.15.32:0/3688304180 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fac40095310 con 0x7fac44100920 2023-08-11T15:43:54.915 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:43:54 smithi134 systemd[1]: Starting Ceph mds.e for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:43:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.094+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac4410c400 con 0x7fac44100920 2023-08-11T15:43:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.095+0000 7fac3affd700 1 -- 172.21.15.32:0/3688304180 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7fac40098930 con 0x7fac44100920 2023-08-11T15:43:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:55.097 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-11T15:43:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fac3009ffa0 msgr2=0x7fac300a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fac3009ffa0 0x7fac300a2460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac34007be0 tx=0x7fac340073d0 comp rx=0 tx=0).stop 2023-08-11T15:43:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac44100920 msgr2=0x7fac441988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac44100920 0x7fac441988d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac4000e970 tx=0x7fac4000ec80 comp rx=0 tx=0).stop 2023-08-11T15:43:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 shutdown_connections 2023-08-11T15:43:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fac3009ffa0 0x7fac300a2460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac441024b0 0x7fac4419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac44100920 0x7fac441988d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 --2- 172.21.15.32:0/3688304180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac44101b10 0x7fac44198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 >> 172.21.15.32:0/3688304180 conn(0x7fac440fbe50 msgr2=0x7fac44107360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 shutdown_connections 2023-08-11T15:43:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:55.098+0000 7fac4c1ff700 1 -- 172.21.15.32:0/3688304180 wait complete. 2023-08-11T15:43:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2023-08-11T15:43:55.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:55 smithi134 ceph-mon[130447]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:55.167 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:43:54 smithi134 podman[154823]: 2023-08-11T15:43:55.167 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:43:55 smithi134 ceph-mds[154863]: -- 172.21.15.134:0/1293966710 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56062bd5b1e0 con 0x56062bd31000 2023-08-11T15:43:55.167 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:43:55 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-e[154839]: starting mds.e at 2023-08-11T15:43:55.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:55 smithi032 ceph-mon[97103]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:55.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:55 smithi192 ceph-mon[130806]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:55.470 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:43:55 smithi134 bash[154823]: f7dce63560e925d2ece7481a3499b204ed80a35b639e65ca0f7f43dea68d819b 2023-08-11T15:43:55.471 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:43:55 smithi134 systemd[1]: Started Ceph mds.e for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:43:56.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3688304180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:56.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:56.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:56.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:56.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:56 smithi192 ceph-mon[130806]: Deploying daemon mds.f on smithi192 2023-08-11T15:43:56.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3688304180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:56.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:56.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:56.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:56 smithi032 ceph-mon[97103]: Deploying daemon mds.f on smithi192 2023-08-11T15:43:56.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3688304180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:56.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:56 smithi134 ceph-mon[130447]: Deploying daemon mds.f on smithi192 2023-08-11T15:43:56.834 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:43:56 smithi192 systemd[1]: Starting Ceph mds.f for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:43:57.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:43:57.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:57 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6834/1559453954,v1:172.21.15.134:6835/1559453954] up:boot 2023-08-11T15:43:57.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:57 smithi032 ceph-mon[97103]: fsmap 5 up:standby 2023-08-11T15:43:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:57 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T15:43:57.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:57 smithi032 ceph-mon[97103]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:57.333 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:43:57 smithi192 podman[156069]: 2023-08-11T15:43:57.334 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:43:57 smithi192 ceph-mds[156109]: -- 172.21.15.192:0/3559166038 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55910c9951e0 con 0x55910c96b000 2023-08-11T15:43:57.334 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:43:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-f[156085]: starting mds.f at 2023-08-11T15:43:57.334 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:43:57 smithi192 bash[156069]: 68c0317b4379fca5c4ec2197545f5c36d0f48d8eb7848c7a69bb702773fb019a 2023-08-11T15:43:57.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:57 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6834/1559453954,v1:172.21.15.134:6835/1559453954] up:boot 2023-08-11T15:43:57.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:57 smithi192 ceph-mon[130806]: fsmap 5 up:standby 2023-08-11T15:43:57.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:57 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T15:43:57.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:57 smithi192 ceph-mon[130806]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.333+0000 7f87883ff700 1 -- 172.21.15.32:0/1606890545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780101b20 msgr2=0x7f8780101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.333+0000 7f87883ff700 1 --2- 172.21.15.32:0/1606890545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780101b20 0x7f8780101f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8770009990 tx=0x7f8770009ca0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.334+0000 7f87883ff700 1 -- 172.21.15.32:0/1606890545 shutdown_connections 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.334+0000 7f87883ff700 1 --2- 172.21.15.32:0/1606890545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8780100930 0x7f8780100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.334+0000 7f87883ff700 1 --2- 172.21.15.32:0/1606890545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87801024c0 0x7f8780108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.334+0000 7f87883ff700 1 --2- 172.21.15.32:0/1606890545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780101b20 0x7f8780101f80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.334+0000 7f87883ff700 1 -- 172.21.15.32:0/1606890545 >> 172.21.15.32:0/1606890545 conn(0x7f87800fbe60 msgr2=0x7f87800fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.334+0000 7f87883ff700 1 -- 172.21.15.32:0/1606890545 shutdown_connections 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.334+0000 7f87883ff700 1 -- 172.21.15.32:0/1606890545 wait complete. 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 Processor -- start 2023-08-11T15:43:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 -- start start 2023-08-11T15:43:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780100930 0x7f87801987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8780101b20 0x7f8780198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87801024c0 0x7f878019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87801a0360 con 0x7f87801024c0 2023-08-11T15:43:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87801a04a0 con 0x7f8780101b20 2023-08-11T15:43:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f87883ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87801a0630 con 0x7f8780100930 2023-08-11T15:43:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f878699c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87801024c0 0x7f878019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f878619b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780100930 0x7f87801987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f878699c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87801024c0 0x7f878019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43594/0 (socket says 172.21.15.32:43594) 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f878619b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780100930 0x7f87801987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43246/0 (socket says 172.21.15.32:43246) 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.335+0000 7f878699c700 1 -- 172.21.15.32:0/925188181 learned_addr learned my addr 172.21.15.32:0/925188181 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878599a700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8780101b20 0x7f8780198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878619b700 1 -- 172.21.15.32:0/925188181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8780101b20 msgr2=0x7f8780198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878619b700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8780101b20 0x7f8780198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878619b700 1 -- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87801024c0 msgr2=0x7f878019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878619b700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87801024c0 0x7f878019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878619b700 1 -- 172.21.15.32:0/925188181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700095f0 con 0x7f8780100930 2023-08-11T15:43:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878699c700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87801024c0 0x7f878019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:43:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f878619b700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780100930 0x7f87801987e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f877c00e9c0 tx=0x7f877c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f87777fe700 1 -- 172.21.15.32:0/925188181 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c004220 con 0x7f8780100930 2023-08-11T15:43:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f87777fe700 1 -- 172.21.15.32:0/925188181 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f877c004380 con 0x7f8780100930 2023-08-11T15:43:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87801a0910 con 0x7f8780100930 2023-08-11T15:43:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f87777fe700 1 -- 172.21.15.32:0/925188181 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c003be0 con 0x7f8780100930 2023-08-11T15:43:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.336+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87801a0e20 con 0x7f8780100930 2023-08-11T15:43:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.337+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878010c190 con 0x7f8780100930 2023-08-11T15:43:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.339+0000 7f87777fe700 1 -- 172.21.15.32:0/925188181 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f877c003d40 con 0x7f8780100930 2023-08-11T15:43:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.341+0000 7f87777fe700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f876c09ffa0 0x7f876c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.341+0000 7f87777fe700 1 -- 172.21.15.32:0/925188181 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f877c014070 con 0x7f8780100930 2023-08-11T15:43:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.341+0000 7f878599a700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f876c09ffa0 0x7f876c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.341+0000 7f878599a700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f876c09ffa0 0x7f876c0a2460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8770006010 tx=0x7f8770005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.343+0000 7f87777fe700 1 -- 172.21.15.32:0/925188181 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f877c0953e0 con 0x7f8780100930 2023-08-11T15:43:57.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:57 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6834/1559453954,v1:172.21.15.134:6835/1559453954] up:boot 2023-08-11T15:43:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:57 smithi134 ceph-mon[130447]: fsmap 5 up:standby 2023-08-11T15:43:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:57 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T15:43:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:57 smithi134 ceph-mon[130447]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:57.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.606+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f878010c350 con 0x7f8780100930 2023-08-11T15:43:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.607+0000 7f87777fe700 1 -- 172.21.15.32:0/925188181 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7f877c098a00 con 0x7f8780100930 2023-08-11T15:43:57.608 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:43:57.609 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-08-11T15:43:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f876c09ffa0 msgr2=0x7f876c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f876c09ffa0 0x7f876c0a2460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8770006010 tx=0x7f8770005a90 comp rx=0 tx=0).stop 2023-08-11T15:43:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780100930 msgr2=0x7f87801987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:57.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780100930 0x7f87801987e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f877c00e9c0 tx=0x7f877c00ecd0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 shutdown_connections 2023-08-11T15:43:57.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f876c09ffa0 0x7f876c0a2460 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87801024c0 0x7f878019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8780100930 0x7f87801987e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 --2- 172.21.15.32:0/925188181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8780101b20 0x7f8780198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.609+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 >> 172.21.15.32:0/925188181 conn(0x7f87800fbe60 msgr2=0x7f87800fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.610+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 shutdown_connections 2023-08-11T15:43:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:57.610+0000 7f87883ff700 1 -- 172.21.15.32:0/925188181 wait complete. 2023-08-11T15:43:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2023-08-11T15:43:57.833 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:43:57 smithi192 systemd[1]: Started Ceph mds.f for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:43:58.665 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:58.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:58.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:58.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: Deploying daemon mds.c on smithi192 2023-08-11T15:43:58.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/925188181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:58.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.192:6834/3482775903,v1:172.21.15.192:6835/3482775903] up:boot 2023-08-11T15:43:58.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: fsmap 6 up:standby 2023-08-11T15:43:58.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T15:43:58.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:43:58 smithi192 ceph-mon[130806]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:58.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:58.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:58.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:58.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: Deploying daemon mds.c on smithi192 2023-08-11T15:43:58.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/925188181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:58.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.192:6834/3482775903,v1:172.21.15.192:6835/3482775903] up:boot 2023-08-11T15:43:58.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: fsmap 6 up:standby 2023-08-11T15:43:58.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T15:43:58.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:43:58 smithi134 ceph-mon[130447]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:58.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:43:58.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:43:58.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:43:58.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:43:58.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: Deploying daemon mds.c on smithi192 2023-08-11T15:43:58.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/925188181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:43:58.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.192:6834/3482775903,v1:172.21.15.192:6835/3482775903] up:boot 2023-08-11T15:43:58.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: fsmap 6 up:standby 2023-08-11T15:43:58.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T15:43:58.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:43:58 smithi032 ceph-mon[97103]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:43:58.915 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:43:58 smithi192 systemd[1]: Starting Ceph mds.c for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:43:59.478 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:43:59 smithi192 podman[156409]: 2023-08-11T15:43:59.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:43:59.834 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:43:59 smithi192 ceph-mds[156449]: -- 172.21.15.192:0/2509102799 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5616804b91e0 con 0x56168048f000 2023-08-11T15:43:59.834 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:43:59 smithi192 bash[156409]: 47959591fee3d09bfbdda01e953652398baff0456c5b4bd423ad546005d0228e 2023-08-11T15:43:59.834 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:43:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-c[156425]: starting mds.c at 2023-08-11T15:43:59.834 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:43:59 smithi192 systemd[1]: Started Ceph mds.c for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:43:59.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 -- 172.21.15.32:0/883344591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c100930 msgr2=0x7f1c7c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 --2- 172.21.15.32:0/883344591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c100930 0x7f1c7c100d50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009920 tx=0x7f1c6c009c30 comp rx=0 tx=0).stop 2023-08-11T15:43:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 -- 172.21.15.32:0/883344591 shutdown_connections 2023-08-11T15:43:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 --2- 172.21.15.32:0/883344591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c1024c0 0x7f1c7c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 --2- 172.21.15.32:0/883344591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c100930 0x7f1c7c100d50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 --2- 172.21.15.32:0/883344591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c7c101b20 0x7f1c7c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:59.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 -- 172.21.15.32:0/883344591 >> 172.21.15.32:0/883344591 conn(0x7f1c7c0fbe60 msgr2=0x7f1c7c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:43:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 -- 172.21.15.32:0/883344591 shutdown_connections 2023-08-11T15:43:59.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.844+0000 7f1c827ff700 1 -- 172.21.15.32:0/883344591 wait complete. 2023-08-11T15:43:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.845+0000 7f1c827ff700 1 Processor -- start 2023-08-11T15:43:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.845+0000 7f1c827ff700 1 -- start start 2023-08-11T15:43:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c827ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c100930 0x7f1c7c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c827ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c7c101b20 0x7f1c7c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c827ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c1024c0 0x7f1c7c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c827ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a04b0 con 0x7f1c7c100930 2023-08-11T15:43:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c827ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a05f0 con 0x7f1c7c1024c0 2023-08-11T15:43:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c827ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a0780 con 0x7f1c7c101b20 2023-08-11T15:43:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c100930 0x7f1c7c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c100930 0x7f1c7c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43604/0 (socket says 172.21.15.32:43604) 2023-08-11T15:43:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7bfff700 1 -- 172.21.15.32:0/3250792542 learned_addr learned my addr 172.21.15.32:0/3250792542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7b7fe700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c7c101b20 0x7f1c7c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c80d9c700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c1024c0 0x7f1c7c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7bfff700 1 -- 172.21.15.32:0/3250792542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c7c101b20 msgr2=0x7f1c7c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7bfff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c7c101b20 0x7f1c7c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7bfff700 1 -- 172.21.15.32:0/3250792542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c1024c0 msgr2=0x7f1c7c19fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.846+0000 7f1c7bfff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c1024c0 0x7f1c7c19fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.847+0000 7f1c7bfff700 1 -- 172.21.15.32:0/3250792542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74009d00 con 0x7f1c7c100930 2023-08-11T15:43:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.847+0000 7f1c7bfff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c100930 0x7f1c7c198930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009230 tx=0x7f1c6c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.847+0000 7f1c797fa700 1 -- 172.21.15.32:0/3250792542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c01c070 con 0x7f1c7c100930 2023-08-11T15:43:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.847+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c009520 con 0x7f1c7c100930 2023-08-11T15:43:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.847+0000 7f1c797fa700 1 -- 172.21.15.32:0/3250792542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c6c00f7d0 con 0x7f1c7c100930 2023-08-11T15:43:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.847+0000 7f1c797fa700 1 -- 172.21.15.32:0/3250792542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c004a00 con 0x7f1c7c100930 2023-08-11T15:43:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.847+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c1a0d10 con 0x7f1c7c100930 2023-08-11T15:43:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.848+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c10c210 con 0x7f1c7c100930 2023-08-11T15:43:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.850+0000 7f1c797fa700 1 -- 172.21.15.32:0/3250792542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f1c6c020070 con 0x7f1c7c100930 2023-08-11T15:43:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.851+0000 7f1c797fa700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1c6409ffa0 0x7f1c640a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:43:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.852+0000 7f1c797fa700 1 -- 172.21.15.32:0/3250792542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c6c0cbf20 con 0x7f1c7c100930 2023-08-11T15:43:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.852+0000 7f1c7b7fe700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1c6409ffa0 0x7f1c640a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:43:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.852+0000 7f1c7b7fe700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1c6409ffa0 0x7f1c640a2460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009a10 tx=0x7f1c70009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:43:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:43:59.854+0000 7f1c797fa700 1 -- 172.21.15.32:0/3250792542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1c6c096140 con 0x7f1c7c100930 2023-08-11T15:44:00.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.113+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c7c10c400 con 0x7f1c7c100930 2023-08-11T15:44:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.114+0000 7f1c797fa700 1 -- 172.21.15.32:0/3250792542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f1c6c099760 con 0x7f1c7c100930 2023-08-11T15:44:00.117 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:00.117 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-08-11T15:44:00.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1c6409ffa0 msgr2=0x7f1c640a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:00.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1c6409ffa0 0x7f1c640a2460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009a10 tx=0x7f1c70009190 comp rx=0 tx=0).stop 2023-08-11T15:44:00.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c100930 msgr2=0x7f1c7c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:00.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c100930 0x7f1c7c198930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009230 tx=0x7f1c6c00b790 comp rx=0 tx=0).stop 2023-08-11T15:44:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 shutdown_connections 2023-08-11T15:44:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1c6409ffa0 0x7f1c640a2460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c100930 0x7f1c7c198930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c7c1024c0 0x7f1c7c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:00.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 --2- 172.21.15.32:0/3250792542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c7c101b20 0x7f1c7c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:00.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 >> 172.21.15.32:0/3250792542 conn(0x7f1c7c0fbe60 msgr2=0x7f1c7c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:00.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 shutdown_connections 2023-08-11T15:44:00.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:00.116+0000 7f1c827ff700 1 -- 172.21.15.32:0/3250792542 wait complete. 2023-08-11T15:44:00.122 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-08-11T15:44:00.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:00.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:00.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:00.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: Deploying daemon mds.h on smithi134 2023-08-11T15:44:00.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.192:6836/3370727785,v1:172.21.15.192:6837/3370727785] up:boot 2023-08-11T15:44:00.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: fsmap 7 up:standby 2023-08-11T15:44:00.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T15:44:00.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:00.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:00 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3250792542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:00.807 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.807 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.808 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.808 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:00.809 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:00.809 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:00.809 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: Deploying daemon mds.h on smithi134 2023-08-11T15:44:00.809 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.192:6836/3370727785,v1:172.21.15.192:6837/3370727785] up:boot 2023-08-11T15:44:00.810 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: fsmap 7 up:standby 2023-08-11T15:44:00.810 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T15:44:00.810 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:00.810 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:00 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3250792542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:00.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: Deploying daemon mds.h on smithi134 2023-08-11T15:44:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.192:6836/3370727785,v1:172.21.15.192:6837/3370727785] up:boot 2023-08-11T15:44:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: fsmap 7 up:standby 2023-08-11T15:44:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T15:44:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:00.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:00 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3250792542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:01.220 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:44:01 smithi134 systemd[1]: Starting Ceph mds.h for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:44:01.557 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:44:01 smithi134 podman[155165]: 2023-08-11T15:44:01.970 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:44:01 smithi134 bash[155165]: c7bd13c5307e14d05184280b9aad5b7d50e9ce53da69961fe20c2f46cfed2957 2023-08-11T15:44:01.971 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:44:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: starting mds.h at 2023-08-11T15:44:01.971 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:44:01 smithi134 systemd[1]: Started Ceph mds.h for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:44:02.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:02.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.354+0000 7f6cb67ff700 1 -- 172.21.15.32:0/1665090637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb01024b0 msgr2=0x7f6cb0108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.354+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/1665090637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb01024b0 0x7f6cb0108d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800b0d0 tx=0x7f6ca800b3e0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.355+0000 7f6cb67ff700 1 -- 172.21.15.32:0/1665090637 shutdown_connections 2023-08-11T15:44:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.355+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/1665090637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb01024b0 0x7f6cb0108d60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.355+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/1665090637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb0100920 0x7f6cb0100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.355+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/1665090637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb0101b10 0x7f6cb0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.355+0000 7f6cb67ff700 1 -- 172.21.15.32:0/1665090637 >> 172.21.15.32:0/1665090637 conn(0x7f6cb00fbe50 msgr2=0x7f6cb00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.355+0000 7f6cb67ff700 1 -- 172.21.15.32:0/1665090637 shutdown_connections 2023-08-11T15:44:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.355+0000 7f6cb67ff700 1 -- 172.21.15.32:0/1665090637 wait complete. 2023-08-11T15:44:02.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 Processor -- start 2023-08-11T15:44:02.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 -- start start 2023-08-11T15:44:02.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb0100920 0x7f6cb00746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:02.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb0101b10 0x7f6cb0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:02.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb01024b0 0x7f6cb0073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:02.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00739f0 con 0x7f6cb0101b10 2023-08-11T15:44:02.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0073b60 con 0x7f6cb01024b0 2023-08-11T15:44:02.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6cb67ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb01a0730 con 0x7f6cb0100920 2023-08-11T15:44:02.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.356+0000 7f6caf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb0101b10 0x7f6cb0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb4d9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb01024b0 0x7f6cb0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6caf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb0101b10 0x7f6cb0072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43630/0 (socket says 172.21.15.32:43630) 2023-08-11T15:44:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6caf7fe700 1 -- 172.21.15.32:0/974678873 learned_addr learned my addr 172.21.15.32:0/974678873 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6caffff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb0100920 0x7f6cb00746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb4d9c700 1 -- 172.21.15.32:0/974678873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb0100920 msgr2=0x7f6cb00746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb4d9c700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb0100920 0x7f6cb00746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb4d9c700 1 -- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb0101b10 msgr2=0x7f6cb0072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb4d9c700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb0101b10 0x7f6cb0072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb4d9c700 1 -- 172.21.15.32:0/974678873 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca0009520 con 0x7f6cb01024b0 2023-08-11T15:44:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6caf7fe700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb0101b10 0x7f6cb0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb4d9c700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb01024b0 0x7f6cb0073250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8007a30 tx=0x7f6ca8008eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cad7fa700 1 -- 172.21.15.32:0/974678873 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800e040 con 0x7f6cb01024b0 2023-08-11T15:44:02.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca8009d00 con 0x7f6cb01024b0 2023-08-11T15:44:02.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cad7fa700 1 -- 172.21.15.32:0/974678873 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ca800bd30 con 0x7f6cb01024b0 2023-08-11T15:44:02.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.357+0000 7f6cad7fa700 1 -- 172.21.15.32:0/974678873 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca801d470 con 0x7f6cb01024b0 2023-08-11T15:44:02.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.358+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb01a0cf0 con 0x7f6cb01024b0 2023-08-11T15:44:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.358+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb0107e10 con 0x7f6cb01024b0 2023-08-11T15:44:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.361+0000 7f6cad7fa700 1 -- 172.21.15.32:0/974678873 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f6ca8005020 con 0x7f6cb01024b0 2023-08-11T15:44:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.362+0000 7f6cad7fa700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c980a0020 0x7f6c980a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.362+0000 7f6cad7fa700 1 -- 172.21.15.32:0/974678873 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ca80cbf50 con 0x7f6cb01024b0 2023-08-11T15:44:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.362+0000 7f6caffff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c980a0020 0x7f6c980a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.363+0000 7f6caffff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c980a0020 0x7f6c980a24e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0101970 tx=0x7f6ca0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.364+0000 7f6cad7fa700 1 -- 172.21.15.32:0/974678873 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6ca8096170 con 0x7f6cb01024b0 2023-08-11T15:44:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.623+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cb002ce20 con 0x7f6cb01024b0 2023-08-11T15:44:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.624+0000 7f6cad7fa700 1 -- 172.21.15.32:0/974678873 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f6ca8099790 con 0x7f6cb01024b0 2023-08-11T15:44:02.626 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:02.626 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-08-11T15:44:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c980a0020 msgr2=0x7f6c980a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c980a0020 0x7f6c980a24e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0101970 tx=0x7f6ca0009260 comp rx=0 tx=0).stop 2023-08-11T15:44:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb01024b0 msgr2=0x7f6cb0073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb01024b0 0x7f6cb0073250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8007a30 tx=0x7f6ca8008eb0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 shutdown_connections 2023-08-11T15:44:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6c980a0020 0x7f6c980a24e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb01024b0 0x7f6cb0073250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb0100920 0x7f6cb00746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 --2- 172.21.15.32:0/974678873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cb0101b10 0x7f6cb0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 >> 172.21.15.32:0/974678873 conn(0x7f6cb00fbe50 msgr2=0x7f6cb0104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 shutdown_connections 2023-08-11T15:44:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:02.626+0000 7f6cb67ff700 1 -- 172.21.15.32:0/974678873 wait complete. 2023-08-11T15:44:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-08-11T15:44:02.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:02.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:02.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:02.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:02.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:02.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:02.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: Deploying daemon mds.i on smithi192 2023-08-11T15:44:02.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6836/4150364023,v1:172.21.15.134:6837/4150364023] up:boot 2023-08-11T15:44:02.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: fsmap 8 up:standby 2023-08-11T15:44:02.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:44:02.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:02.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:02 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/974678873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:02.989 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:02.989 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:02.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:02.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:02.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:02.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:02.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: Deploying daemon mds.i on smithi192 2023-08-11T15:44:02.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6836/4150364023,v1:172.21.15.134:6837/4150364023] up:boot 2023-08-11T15:44:02.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: fsmap 8 up:standby 2023-08-11T15:44:02.991 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:44:02.991 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:02.991 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:02 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/974678873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:03.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:03.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:03.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:03.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:03.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:03.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:03.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: Deploying daemon mds.i on smithi192 2023-08-11T15:44:03.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6836/4150364023,v1:172.21.15.134:6837/4150364023] up:boot 2023-08-11T15:44:03.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: fsmap 8 up:standby 2023-08-11T15:44:03.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:44:03.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:03.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:02 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/974678873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:03.239 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:44:03 smithi192 systemd[1]: Starting Ceph mds.i for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:44:03.778 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:44:03 smithi192 podman[156746]: 2023-08-11T15:44:03.779 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:44:03 smithi192 ceph-mds[156786]: -- 172.21.15.192:0/1136180370 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5608df8971e0 con 0x5608df86d000 2023-08-11T15:44:03.779 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:44:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-i[156762]: starting mds.i at 2023-08-11T15:44:04.083 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:44:03 smithi192 bash[156746]: 9372634112bf26f5ea98487a193282dcaa4c1dd4f29f58cd0c86cc31b1fae98e 2023-08-11T15:44:04.083 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:44:03 smithi192 systemd[1]: Started Ceph mds.i for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:44:04.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 -- 172.21.15.32:0/2867521440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 msgr2=0x7f2d5c103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 --2- 172.21.15.32:0/2867521440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 0x7f2d5c103780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009990 tx=0x7f2d4c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 -- 172.21.15.32:0/2867521440 shutdown_connections 2023-08-11T15:44:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 --2- 172.21.15.32:0/2867521440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c1065f0 0x7f2d5c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 --2- 172.21.15.32:0/2867521440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 0x7f2d5c103780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 --2- 172.21.15.32:0/2867521440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d5c103cc0 0x7f2d5c1060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 -- 172.21.15.32:0/2867521440 >> 172.21.15.32:0/2867521440 conn(0x7f2d5c0fad80 msgr2=0x7f2d5c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 -- 172.21.15.32:0/2867521440 shutdown_connections 2023-08-11T15:44:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.873+0000 7f2d621ff700 1 -- 172.21.15.32:0/2867521440 wait complete. 2023-08-11T15:44:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.874+0000 7f2d621ff700 1 Processor -- start 2023-08-11T15:44:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.875+0000 7f2d621ff700 1 -- start start 2023-08-11T15:44:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.875+0000 7f2d621ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 0x7f2d5c100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.875+0000 7f2d621ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c103cc0 0x7f2d5c0ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.875+0000 7f2d621ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d5c1065f0 0x7f2d5c0ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.875+0000 7f2d621ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c100020 con 0x7f2d5c103cc0 2023-08-11T15:44:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.875+0000 7f2d621ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c100190 con 0x7f2d5c101390 2023-08-11T15:44:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.875+0000 7f2d621ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c1a05b0 con 0x7f2d5c1065f0 2023-08-11T15:44:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c103cc0 0x7f2d5c0ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c103cc0 0x7f2d5c0ff340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49774/0 (socket says 172.21.15.32:49774) 2023-08-11T15:44:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d5c1065f0 0x7f2d5c0ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 -- 172.21.15.32:0/3232181462 learned_addr learned my addr 172.21.15.32:0/3232181462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5b7fe700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 0x7f2d5c100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 -- 172.21.15.32:0/3232181462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d5c1065f0 msgr2=0x7f2d5c0ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d5c1065f0 0x7f2d5c0ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 -- 172.21.15.32:0/3232181462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 msgr2=0x7f2d5c100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 0x7f2d5c100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 -- 172.21.15.32:0/3232181462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0095f0 con 0x7f2d5c103cc0 2023-08-11T15:44:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5bfff700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d5c1065f0 0x7f2d5c0ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.876+0000 7f2d5affd700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c103cc0 0x7f2d5c0ff340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9a0 tx=0x7f2d5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.877+0000 7f2d58ff9700 1 -- 172.21.15.32:0/3232181462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50009790 con 0x7f2d5c103cc0 2023-08-11T15:44:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.877+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c1a0890 con 0x7f2d5c103cc0 2023-08-11T15:44:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.877+0000 7f2d58ff9700 1 -- 172.21.15.32:0/3232181462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d50010430 con 0x7f2d5c103cc0 2023-08-11T15:44:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.877+0000 7f2d58ff9700 1 -- 172.21.15.32:0/3232181462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50018680 con 0x7f2d5c103cc0 2023-08-11T15:44:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.877+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c1a0da0 con 0x7f2d5c103cc0 2023-08-11T15:44:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.878+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5c10c0f0 con 0x7f2d5c103cc0 2023-08-11T15:44:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.881+0000 7f2d58ff9700 1 -- 172.21.15.32:0/3232181462 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f2d500105a0 con 0x7f2d5c103cc0 2023-08-11T15:44:04.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.882+0000 7f2d58ff9700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2d440a0020 0x7f2d440a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.882+0000 7f2d58ff9700 1 -- 172.21.15.32:0/3232181462 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2d50014070 con 0x7f2d5c103cc0 2023-08-11T15:44:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.883+0000 7f2d5b7fe700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2d440a0020 0x7f2d440a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.884+0000 7f2d5b7fe700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2d440a0020 0x7f2d440a24e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00b680 tx=0x7f2d4c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:04.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:04.885+0000 7f2d58ff9700 1 -- 172.21.15.32:0/3232181462 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2d50095380 con 0x7f2d5c103cc0 2023-08-11T15:44:04.990 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:04.990 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:04.991 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:04.991 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:04.991 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:04.991 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:04.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: Deploying daemon mds.g on smithi032 2023-08-11T15:44:04.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.192:6838/2697851094,v1:172.21.15.192:6839/2697851094] up:boot 2023-08-11T15:44:04.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: fsmap 9 up:standby 2023-08-11T15:44:04.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T15:44:04.993 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:04 smithi032 ceph-mon[97103]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:05.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.148+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d5c10c330 con 0x7f2d5c103cc0 2023-08-11T15:44:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.149+0000 7f2d58ff9700 1 -- 172.21.15.32:0/3232181462 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f2d500989a0 con 0x7f2d5c103cc0 2023-08-11T15:44:05.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:05.152 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-08-11T15:44:05.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2d440a0020 msgr2=0x7f2d440a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2d440a0020 0x7f2d440a24e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00b680 tx=0x7f2d4c005fd0 comp rx=0 tx=0).stop 2023-08-11T15:44:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c103cc0 msgr2=0x7f2d5c0ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c103cc0 0x7f2d5c0ff340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9a0 tx=0x7f2d5000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 shutdown_connections 2023-08-11T15:44:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2d440a0020 0x7f2d440a24e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d5c1065f0 0x7f2d5c0ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c101390 0x7f2d5c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 --2- 172.21.15.32:0/3232181462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d5c103cc0 0x7f2d5c0ff340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 >> 172.21.15.32:0/3232181462 conn(0x7f2d5c0fad80 msgr2=0x7f2d5c1027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 shutdown_connections 2023-08-11T15:44:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:05.153+0000 7f2d621ff700 1 -- 172.21.15.32:0/3232181462 wait complete. 2023-08-11T15:44:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2023-08-11T15:44:05.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:05.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:05.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: Deploying daemon mds.g on smithi032 2023-08-11T15:44:05.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.192:6838/2697851094,v1:172.21.15.192:6839/2697851094] up:boot 2023-08-11T15:44:05.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: fsmap 9 up:standby 2023-08-11T15:44:05.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T15:44:05.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:04 smithi134 ceph-mon[130447]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:05.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:05.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: Deploying daemon mds.g on smithi032 2023-08-11T15:44:05.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.192:6838/2697851094,v1:172.21.15.192:6839/2697851094] up:boot 2023-08-11T15:44:05.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: fsmap 9 up:standby 2023-08-11T15:44:05.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T15:44:05.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:04 smithi192 ceph-mon[130806]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:06.044 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:06 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3232181462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:06.044 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:44:06 smithi032 systemd[1]: Starting Ceph mds.g for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:44:06.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:06 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3232181462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:06.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:06 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3232181462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:06.779 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:44:06 smithi032 podman[130735]: 2023-08-11T15:44:06.779 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:44:06 smithi032 bash[130735]: 85b2f72b4dfefe284bc50ad685a9959f52fd454af71d5842b0ffff48086e68c9 2023-08-11T15:44:07.048 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:44:06 smithi032 ceph-mds[130775]: -- 172.21.15.32:0/1964507274 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568486071e0 con 0x5568485dd000 2023-08-11T15:44:07.048 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:44:06 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-g[130751]: starting mds.g at 2023-08-11T15:44:07.048 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:44:06 smithi032 systemd[1]: Started Ceph mds.g for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:44:07.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:07 smithi032 ceph-mon[97103]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:07.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:07 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:07.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:07 smithi192 ceph-mon[130806]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:07.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:07 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 -- 172.21.15.32:0/1432772719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec9810cef0 msgr2=0x7fec9810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 --2- 172.21.15.32:0/1432772719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec9810cef0 0x7fec9810f2e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec9000b0a0 tx=0x7fec9000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 -- 172.21.15.32:0/1432772719 shutdown_connections 2023-08-11T15:44:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 --2- 172.21.15.32:0/1432772719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 0x7fec98072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 --2- 172.21.15.32:0/1432772719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec9810cef0 0x7fec9810f2e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 --2- 172.21.15.32:0/1432772719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec98073380 0x7fec98073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 -- 172.21.15.32:0/1432772719 >> 172.21.15.32:0/1432772719 conn(0x7fec980fbe50 msgr2=0x7fec980fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 -- 172.21.15.32:0/1432772719 shutdown_connections 2023-08-11T15:44:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.389+0000 7feca05ff700 1 -- 172.21.15.32:0/1432772719 wait complete. 2023-08-11T15:44:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 Processor -- start 2023-08-11T15:44:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 -- start start 2023-08-11T15:44:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98073380 0x7fec9819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 0x7fec9819d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9810cef0 0x7fec981a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec981a4840 con 0x7fec98074960 2023-08-11T15:44:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec981a49b0 con 0x7fec98073380 2023-08-11T15:44:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.390+0000 7feca05ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec981a4b70 con 0x7fec9810cef0 2023-08-11T15:44:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 0x7fec9819d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9e39b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98073380 0x7fec9819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 0x7fec9819d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49814/0 (socket says 172.21.15.32:49814) 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 -- 172.21.15.32:0/941922575 learned_addr learned my addr 172.21.15.32:0/941922575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9eb9c700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9810cef0 0x7fec981a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 -- 172.21.15.32:0/941922575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9810cef0 msgr2=0x7fec981a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9810cef0 0x7fec981a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 -- 172.21.15.32:0/941922575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98073380 msgr2=0x7fec9819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98073380 0x7fec9819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 -- 172.21.15.32:0/941922575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009d00 con 0x7fec98074960 2023-08-11T15:44:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.391+0000 7fec9db9a700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 0x7fec9819d240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e970 tx=0x7fec8800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.392+0000 7fec8f7fe700 1 -- 172.21.15.32:0/941922575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88009790 con 0x7fec98074960 2023-08-11T15:44:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.392+0000 7fec8f7fe700 1 -- 172.21.15.32:0/941922575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec8800c7e0 con 0x7fec98074960 2023-08-11T15:44:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.392+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec981a4e50 con 0x7fec98074960 2023-08-11T15:44:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.392+0000 7fec8f7fe700 1 -- 172.21.15.32:0/941922575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88003c70 con 0x7fec98074960 2023-08-11T15:44:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.392+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec981a52d0 con 0x7fec98074960 2023-08-11T15:44:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.393+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec98102a40 con 0x7fec98074960 2023-08-11T15:44:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.396+0000 7fec8f7fe700 1 -- 172.21.15.32:0/941922575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fec8800cc50 con 0x7fec98074960 2023-08-11T15:44:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.398+0000 7fec8f7fe700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec840a0020 0x7fec840a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.398+0000 7fec8f7fe700 1 -- 172.21.15.32:0/941922575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fec88014070 con 0x7fec98074960 2023-08-11T15:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.398+0000 7fec9e39b700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec840a0020 0x7fec840a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.399+0000 7fec9e39b700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec840a0020 0x7fec840a24e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fec9819e1d0 tx=0x7fec94009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.402+0000 7fec8f7fe700 1 -- 172.21.15.32:0/941922575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fec88095120 con 0x7fec98074960 2023-08-11T15:44:07.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:07 smithi134 ceph-mon[130447]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:07.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:07 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.688+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec98112760 con 0x7fec98074960 2023-08-11T15:44:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.689+0000 7fec8f7fe700 1 -- 172.21.15.32:0/941922575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7fec88003dd0 con 0x7fec98074960 2023-08-11T15:44:07.692 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:07.692 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-08-11T15:44:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec840a0020 msgr2=0x7fec840a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec840a0020 0x7fec840a24e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fec9819e1d0 tx=0x7fec94009260 comp rx=0 tx=0).stop 2023-08-11T15:44:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 msgr2=0x7fec9819d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 0x7fec9819d240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e970 tx=0x7fec8800ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 shutdown_connections 2023-08-11T15:44:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec840a0020 0x7fec840a24e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9810cef0 0x7fec981a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98073380 0x7fec9819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 --2- 172.21.15.32:0/941922575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec98074960 0x7fec9819d240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 >> 172.21.15.32:0/941922575 conn(0x7fec980fbe50 msgr2=0x7fec980fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.691+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 shutdown_connections 2023-08-11T15:44:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:07.692+0000 7feca05ff700 1 -- 172.21.15.32:0/941922575 wait complete. 2023-08-11T15:44:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 11 2023-08-11T15:44:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:08 smithi032 ceph-mon[97103]: Deploying daemon mds.a on smithi032 2023-08-11T15:44:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:08 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6838/840045338,v1:172.21.15.32:6839/840045338] up:boot 2023-08-11T15:44:08.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:08 smithi032 ceph-mon[97103]: fsmap 10 up:standby 2023-08-11T15:44:08.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:08 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T15:44:08.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:08 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/941922575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:08 smithi192 ceph-mon[130806]: Deploying daemon mds.a on smithi032 2023-08-11T15:44:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:08 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6838/840045338,v1:172.21.15.32:6839/840045338] up:boot 2023-08-11T15:44:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:08 smithi192 ceph-mon[130806]: fsmap 10 up:standby 2023-08-11T15:44:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:08 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T15:44:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:08 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/941922575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:08.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:08 smithi134 ceph-mon[130447]: Deploying daemon mds.a on smithi032 2023-08-11T15:44:08.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:08 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6838/840045338,v1:172.21.15.32:6839/840045338] up:boot 2023-08-11T15:44:08.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:08 smithi134 ceph-mon[130447]: fsmap 10 up:standby 2023-08-11T15:44:08.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:08 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T15:44:08.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:08 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/941922575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:09.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:09 smithi032 ceph-mon[97103]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:09.304 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:44:09 smithi032 systemd[1]: Starting Ceph mds.a for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:44:09.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:09 smithi192 ceph-mon[130806]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:09.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:09 smithi134 ceph-mon[130447]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:09.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:09.781 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:44:09 smithi032 podman[131123]: 2023-08-11T15:44:09.781 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:44:09 smithi032 ceph-mds[131163]: -- 172.21.15.32:0/1386743882 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55deeb3f31e0 con 0x55deeb3c8800 2023-08-11T15:44:09.781 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:44:09 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: starting mds.a at 2023-08-11T15:44:09.781 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:44:09 smithi032 bash[131123]: 016a7e821f64f92c1e2df61bf1be6595dab19f3967ff6facf9ff1c10812e4889 2023-08-11T15:44:09.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 -- 172.21.15.32:0/3334122206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9020101b10 msgr2=0x7f9020101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:09.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 --2- 172.21.15.32:0/3334122206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9020101b10 0x7f9020101f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9014009920 tx=0x7f9014009c30 comp rx=0 tx=0).stop 2023-08-11T15:44:09.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 -- 172.21.15.32:0/3334122206 shutdown_connections 2023-08-11T15:44:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 --2- 172.21.15.32:0/3334122206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90201024b0 0x7f9020108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 --2- 172.21.15.32:0/3334122206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020100920 0x7f9020100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 --2- 172.21.15.32:0/3334122206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9020101b10 0x7f9020101f70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 -- 172.21.15.32:0/3334122206 >> 172.21.15.32:0/3334122206 conn(0x7f90200fbe50 msgr2=0x7f90200fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 -- 172.21.15.32:0/3334122206 shutdown_connections 2023-08-11T15:44:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.936+0000 7f90259ff700 1 -- 172.21.15.32:0/3334122206 wait complete. 2023-08-11T15:44:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 Processor -- start 2023-08-11T15:44:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 -- start start 2023-08-11T15:44:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020100920 0x7f90201966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101b10 0x7f9020196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90201024b0 0x7f902019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019e240 con 0x7f90201024b0 2023-08-11T15:44:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019e380 con 0x7f9020100920 2023-08-11T15:44:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.937+0000 7f90259ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019e510 con 0x7f9020101b10 2023-08-11T15:44:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90201024b0 0x7f902019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020100920 0x7f90201966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020100920 0x7f90201966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:32862/0 (socket says 172.21.15.32:32862) 2023-08-11T15:44:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90201024b0 0x7f902019dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49846/0 (socket says 172.21.15.32:49846) 2023-08-11T15:44:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 -- 172.21.15.32:0/3034410916 learned_addr learned my addr 172.21.15.32:0/3034410916 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901e7fc700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101b10 0x7f9020196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 -- 172.21.15.32:0/3034410916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101b10 msgr2=0x7f9020196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101b10 0x7f9020196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 -- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90201024b0 msgr2=0x7f902019dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90201024b0 0x7f902019dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 -- 172.21.15.32:0/3034410916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9018009d00 con 0x7f9020100920 2023-08-11T15:44:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901f7fe700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90201024b0 0x7f902019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f901effd700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020100920 0x7f90201966c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f901000e970 tx=0x7f901000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.938+0000 7f90249fd700 1 -- 172.21.15.32:0/3034410916 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010009790 con 0x7f9020100920 2023-08-11T15:44:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.939+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9014009520 con 0x7f9020100920 2023-08-11T15:44:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.939+0000 7f90249fd700 1 -- 172.21.15.32:0/3034410916 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f901000c7e0 con 0x7f9020100920 2023-08-11T15:44:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.939+0000 7f90249fd700 1 -- 172.21.15.32:0/3034410916 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010003c70 con 0x7f9020100920 2023-08-11T15:44:09.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.939+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902019eb70 con 0x7f9020100920 2023-08-11T15:44:09.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.940+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902010c180 con 0x7f9020100920 2023-08-11T15:44:09.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.942+0000 7f90249fd700 1 -- 172.21.15.32:0/3034410916 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f901000cc50 con 0x7f9020100920 2023-08-11T15:44:09.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.943+0000 7f90249fd700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f900809ffa0 0x7f90080a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:09.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.943+0000 7f90249fd700 1 -- 172.21.15.32:0/3034410916 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9010014070 con 0x7f9020100920 2023-08-11T15:44:09.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.943+0000 7f901e7fc700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f900809ffa0 0x7f90080a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:09.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.944+0000 7f901e7fc700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f900809ffa0 0x7f90080a2460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f901400fa40 tx=0x7f9014019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:09.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:09.945+0000 7f90249fd700 1 -- 172.21.15.32:0/3034410916 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f90100952e0 con 0x7f9020100920 2023-08-11T15:44:10.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:44:09 smithi032 systemd[1]: Started Ceph mds.a for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:44:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.205+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90201078b0 con 0x7f9020100920 2023-08-11T15:44:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.206+0000 7f90249fd700 1 -- 172.21.15.32:0/3034410916 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f9010003dd0 con 0x7f9020100920 2023-08-11T15:44:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:10.208 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-08-11T15:44:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f900809ffa0 msgr2=0x7f90080a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f900809ffa0 0x7f90080a2460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f901400fa40 tx=0x7f9014019040 comp rx=0 tx=0).stop 2023-08-11T15:44:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020100920 msgr2=0x7f90201966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020100920 0x7f90201966c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f901000e970 tx=0x7f901000ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 shutdown_connections 2023-08-11T15:44:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f900809ffa0 0x7f90080a2460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90201024b0 0x7f902019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020100920 0x7f90201966c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 --2- 172.21.15.32:0/3034410916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101b10 0x7f9020196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.208+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 >> 172.21.15.32:0/3034410916 conn(0x7f90200fbe50 msgr2=0x7f90200fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.209+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 shutdown_connections 2023-08-11T15:44:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:10.209+0000 7f90259ff700 1 -- 172.21.15.32:0/3034410916 wait complete. 2023-08-11T15:44:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2023-08-11T15:44:11.172 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:11.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:11.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:11.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: Deploying daemon mds.b on smithi134 2023-08-11T15:44:11.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6840/1060395589,v1:172.21.15.32:6841/1060395589] up:boot 2023-08-11T15:44:11.174 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: fsmap 11 up:standby 2023-08-11T15:44:11.174 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:44:11.174 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:11.174 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:10 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3034410916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:11.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:11.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:11.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:11.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: Deploying daemon mds.b on smithi134 2023-08-11T15:44:11.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6840/1060395589,v1:172.21.15.32:6841/1060395589] up:boot 2023-08-11T15:44:11.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: fsmap 11 up:standby 2023-08-11T15:44:11.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:44:11.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:11.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:10 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3034410916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:11.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:11.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T15:44:11.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T15:44:11.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:11.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: Deploying daemon mds.b on smithi134 2023-08-11T15:44:11.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6840/1060395589,v1:172.21.15.32:6841/1060395589] up:boot 2023-08-11T15:44:11.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: fsmap 11 up:standby 2023-08-11T15:44:11.336 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:44:11.337 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:11.337 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:10 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3034410916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:11.471 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:44:11 smithi134 systemd[1]: Starting Ceph mds.b for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:44:11.962 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:44:11 smithi134 podman[155507]: 2023-08-11T15:44:11.963 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:44:11 smithi134 bash[155507]: 02f482ab81ac113d9e9d31fa0c394778f064cfe64a98c5a7126deb8492792799 2023-08-11T15:44:12.220 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:44:11 smithi134 ceph-mds[155547]: -- 172.21.15.134:0/2918572431 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621da4db1e0 con 0x5621da4b0c00 2023-08-11T15:44:12.221 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:44:11 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b[155523]: starting mds.b at 2023-08-11T15:44:12.221 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:44:11 smithi134 systemd[1]: Started Ceph mds.b for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:44:12.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 -- 172.21.15.32:0/583678873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108074900 msgr2=0x7f2108072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 --2- 172.21.15.32:0/583678873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108074900 0x7f2108072d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098f0 tx=0x7f20fc009c00 comp rx=0 tx=0).stop 2023-08-11T15:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 -- 172.21.15.32:0/583678873 shutdown_connections 2023-08-11T15:44:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 --2- 172.21.15.32:0/583678873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108074900 0x7f2108072d50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 --2- 172.21.15.32:0/583678873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2108108830 0x7f210810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 --2- 172.21.15.32:0/583678873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2108073320 0x7f21080737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 -- 172.21.15.32:0/583678873 >> 172.21.15.32:0/583678873 conn(0x7f21080fbed0 msgr2=0x7f21080fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 -- 172.21.15.32:0/583678873 shutdown_connections 2023-08-11T15:44:12.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.440+0000 7f210fbff700 1 -- 172.21.15.32:0/583678873 wait complete. 2023-08-11T15:44:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 Processor -- start 2023-08-11T15:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 -- start start 2023-08-11T15:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2108073320 0x7f2108198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2108074900 0x7f2108198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108108830 0x7f210819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21081a03f0 con 0x7f2108108830 2023-08-11T15:44:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21081a0530 con 0x7f2108073320 2023-08-11T15:44:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.442+0000 7f210fbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21081a06c0 con 0x7f2108074900 2023-08-11T15:44:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d19a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2108074900 0x7f2108198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d99b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2108073320 0x7f2108198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d19a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2108074900 0x7f2108198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46588/0 (socket says 172.21.15.32:46588) 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d19a700 1 -- 172.21.15.32:0/672429477 learned_addr learned my addr 172.21.15.32:0/672429477 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210e19c700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108108830 0x7f210819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d99b700 1 -- 172.21.15.32:0/672429477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2108074900 msgr2=0x7f2108198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d99b700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2108074900 0x7f2108198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d99b700 1 -- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108108830 msgr2=0x7f210819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d99b700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108108830 0x7f210819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d99b700 1 -- 172.21.15.32:0/672429477 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc009520 con 0x7f2108073320 2023-08-11T15:44:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.443+0000 7f210d99b700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2108073320 0x7f2108198870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009230 tx=0x7f20fc00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.444+0000 7f20faffd700 1 -- 172.21.15.32:0/672429477 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc01c070 con 0x7f2108073320 2023-08-11T15:44:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.444+0000 7f20faffd700 1 -- 172.21.15.32:0/672429477 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20fc00f820 con 0x7f2108073320 2023-08-11T15:44:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.444+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21081a0940 con 0x7f2108073320 2023-08-11T15:44:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.444+0000 7f20faffd700 1 -- 172.21.15.32:0/672429477 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc003a70 con 0x7f2108073320 2023-08-11T15:44:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.444+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21081a0d60 con 0x7f2108073320 2023-08-11T15:44:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.445+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2108106f30 con 0x7f2108073320 2023-08-11T15:44:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.447+0000 7f20faffd700 1 -- 172.21.15.32:0/672429477 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f20fc020030 con 0x7f2108073320 2023-08-11T15:44:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.449+0000 7f20faffd700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f20f409ffa0 0x7f20f40a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.449+0000 7f20faffd700 1 -- 172.21.15.32:0/672429477 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f20fc0cbe70 con 0x7f2108073320 2023-08-11T15:44:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.449+0000 7f210d19a700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f20f409ffa0 0x7f20f40a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.449+0000 7f210d19a700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f20f409ffa0 0x7f20f40a2460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2104005950 tx=0x7f2104009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.451+0000 7f20faffd700 1 -- 172.21.15.32:0/672429477 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f20fc096090 con 0x7f2108073320 2023-08-11T15:44:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.720+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f210802cd90 con 0x7f2108073320 2023-08-11T15:44:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.721+0000 7f20faffd700 1 -- 172.21.15.32:0/672429477 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f20fc0996b0 con 0x7f2108073320 2023-08-11T15:44:12.724 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:12.724 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-08-11T15:44:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f20f409ffa0 msgr2=0x7f20f40a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f20f409ffa0 0x7f20f40a2460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2104005950 tx=0x7f2104009310 comp rx=0 tx=0).stop 2023-08-11T15:44:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2108073320 msgr2=0x7f2108198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2108073320 0x7f2108198870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009230 tx=0x7f20fc00b790 comp rx=0 tx=0).stop 2023-08-11T15:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 shutdown_connections 2023-08-11T15:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f20f409ffa0 0x7f20f40a2460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2108108830 0x7f210819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2108073320 0x7f2108198870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 --2- 172.21.15.32:0/672429477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2108074900 0x7f2108198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 >> 172.21.15.32:0/672429477 conn(0x7f21080fbed0 msgr2=0x7f21080fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 shutdown_connections 2023-08-11T15:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:12.724+0000 7f210fbff700 1 -- 172.21.15.32:0/672429477 wait complete. 2023-08-11T15:44:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2023-08-11T15:44:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:44:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/672429477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:13.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] up:boot 2023-08-11T15:44:13.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: fsmap 12 up:standby 2023-08-11T15:44:13.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:13 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:44:13.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:44:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/672429477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:13.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] up:boot 2023-08-11T15:44:13.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: fsmap 12 up:standby 2023-08-11T15:44:13.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:13 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:44:13.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:13.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:13.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:44:13.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/672429477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:13.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] up:boot 2023-08-11T15:44:13.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: fsmap 12 up:standby 2023-08-11T15:44:13.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:13 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:44:14.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 -- 172.21.15.32:0/3548366971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14100920 msgr2=0x7feb14100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/3548366971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14100920 0x7feb14100d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb100099c0 tx=0x7feb10009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 -- 172.21.15.32:0/3548366971 shutdown_connections 2023-08-11T15:44:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/3548366971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb141024b0 0x7feb14108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/3548366971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14100920 0x7feb14100d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/3548366971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb14101b10 0x7feb14101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 -- 172.21.15.32:0/3548366971 >> 172.21.15.32:0/3548366971 conn(0x7feb140fbe50 msgr2=0x7feb140fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 -- 172.21.15.32:0/3548366971 shutdown_connections 2023-08-11T15:44:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.958+0000 7feb1b5ff700 1 -- 172.21.15.32:0/3548366971 wait complete. 2023-08-11T15:44:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.959+0000 7feb1b5ff700 1 Processor -- start 2023-08-11T15:44:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.959+0000 7feb1b5ff700 1 -- start start 2023-08-11T15:44:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1b5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb14100920 0x7feb141987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1b5ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14101b10 0x7feb14198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1b5ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb141024b0 0x7feb1419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1b5ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb141a0360 con 0x7feb141024b0 2023-08-11T15:44:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1b5ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb141a04a0 con 0x7feb14101b10 2023-08-11T15:44:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1b5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb141a0630 con 0x7feb14100920 2023-08-11T15:44:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb14100920 0x7feb141987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb14100920 0x7feb141987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:55782/0 (socket says 172.21.15.32:55782) 2023-08-11T15:44:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 -- 172.21.15.32:0/2409161320 learned_addr learned my addr 172.21.15.32:0/2409161320 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb19b9c700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb141024b0 0x7feb1419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb18b9a700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14101b10 0x7feb14198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 -- 172.21.15.32:0/2409161320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14101b10 msgr2=0x7feb14198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14101b10 0x7feb14198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 -- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb141024b0 msgr2=0x7feb1419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb141024b0 0x7feb1419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.960+0000 7feb1939b700 1 -- 172.21.15.32:0/2409161320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100095f0 con 0x7feb14100920 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.961+0000 7feb1939b700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb14100920 0x7feb141987e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb10009990 tx=0x7feb1000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.961+0000 7feb0a7fc700 1 -- 172.21.15.32:0/2409161320 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1001d070 con 0x7feb14100920 2023-08-11T15:44:14.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.961+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb141a08b0 con 0x7feb14100920 2023-08-11T15:44:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.961+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb141a0d30 con 0x7feb14100920 2023-08-11T15:44:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.961+0000 7feb0a7fc700 1 -- 172.21.15.32:0/2409161320 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb10005030 con 0x7feb14100920 2023-08-11T15:44:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.961+0000 7feb0a7fc700 1 -- 172.21.15.32:0/2409161320 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10004a90 con 0x7feb14100920 2023-08-11T15:44:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.962+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1410c180 con 0x7feb14100920 2023-08-11T15:44:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.964+0000 7feb0a7fc700 1 -- 172.21.15.32:0/2409161320 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7feb10005410 con 0x7feb14100920 2023-08-11T15:44:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.965+0000 7feb0a7fc700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7feb000a0020 0x7feb000a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.966+0000 7feb0a7fc700 1 -- 172.21.15.32:0/2409161320 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb100cb9c0 con 0x7feb14100920 2023-08-11T15:44:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.966+0000 7feb18b9a700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7feb000a0020 0x7feb000a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.966+0000 7feb18b9a700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7feb000a0020 0x7feb000a24e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb04009c60 tx=0x7feb04009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:14.967+0000 7feb0a7fc700 1 -- 172.21.15.32:0/2409161320 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7feb10095be0 con 0x7feb14100920 2023-08-11T15:44:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.245+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb1410c370 con 0x7feb14100920 2023-08-11T15:44:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.246+0000 7feb0a7fc700 1 -- 172.21.15.32:0/2409161320 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7feb10099200 con 0x7feb14100920 2023-08-11T15:44:15.249 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:15.250 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-08-11T15:44:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7feb000a0020 msgr2=0x7feb000a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7feb000a0020 0x7feb000a24e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb04009c60 tx=0x7feb04009260 comp rx=0 tx=0).stop 2023-08-11T15:44:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb14100920 msgr2=0x7feb141987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb14100920 0x7feb141987e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb10009990 tx=0x7feb1000f690 comp rx=0 tx=0).stop 2023-08-11T15:44:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 shutdown_connections 2023-08-11T15:44:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7feb000a0020 0x7feb000a24e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb141024b0 0x7feb1419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb14100920 0x7feb141987e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 --2- 172.21.15.32:0/2409161320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14101b10 0x7feb14198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 >> 172.21.15.32:0/2409161320 conn(0x7feb140fbe50 msgr2=0x7feb141073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 shutdown_connections 2023-08-11T15:44:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.249+0000 7feb1b5ff700 1 -- 172.21.15.32:0/2409161320 wait complete. 2023-08-11T15:44:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2023-08-11T15:44:15.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:15.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:15 smithi192 ceph-mon[130806]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:15.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:15 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.339 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:15 smithi032 ceph-mon[97103]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:15.339 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.339 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.339 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:15 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:15 smithi134 ceph-mon[130447]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:15.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:15 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 -- 172.21.15.32:0/2654206047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 msgr2=0x7f056c101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 --2- 172.21.15.32:0/2654206047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 0x7f056c101f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0560009990 tx=0x7f0560009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 -- 172.21.15.32:0/2654206047 shutdown_connections 2023-08-11T15:44:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 --2- 172.21.15.32:0/2654206047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 0x7f056c100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 --2- 172.21.15.32:0/2654206047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f056c102440 0x7f056c108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 --2- 172.21.15.32:0/2654206047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 0x7f056c101f00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 -- 172.21.15.32:0/2654206047 >> 172.21.15.32:0/2654206047 conn(0x7f056c0fbe00 msgr2=0x7f056c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 -- 172.21.15.32:0/2654206047 shutdown_connections 2023-08-11T15:44:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.489+0000 7f05713ff700 1 -- 172.21.15.32:0/2654206047 wait complete. 2023-08-11T15:44:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.490+0000 7f05713ff700 1 Processor -- start 2023-08-11T15:44:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.490+0000 7f05713ff700 1 -- start start 2023-08-11T15:44:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f05713ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 0x7f056c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f05713ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 0x7f056c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f05713ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f056c102440 0x7f056c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f05713ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0739f0 con 0x7f056c1008b0 2023-08-11T15:44:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f05713ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c073b60 con 0x7f056c102440 2023-08-11T15:44:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f05713ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c1a0590 con 0x7f056c101aa0 2023-08-11T15:44:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 0x7f056c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 0x7f056c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49896/0 (socket says 172.21.15.32:49896) 2023-08-11T15:44:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056affd700 1 -- 172.21.15.32:0/3983994068 learned_addr learned my addr 172.21.15.32:0/3983994068 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056a7fc700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 0x7f056c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056b7fe700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f056c102440 0x7f056c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056a7fc700 1 -- 172.21.15.32:0/3983994068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f056c102440 msgr2=0x7f056c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056a7fc700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f056c102440 0x7f056c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056a7fc700 1 -- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 msgr2=0x7f056c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056a7fc700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 0x7f056c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056a7fc700 1 -- 172.21.15.32:0/3983994068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600095f0 con 0x7f056c101aa0 2023-08-11T15:44:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.491+0000 7f056affd700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 0x7f056c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.492+0000 7f056a7fc700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 0x7f056c072d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f056000b680 tx=0x7f0560009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.492+0000 7f0553fff700 1 -- 172.21.15.32:0/3983994068 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056001d070 con 0x7f056c101aa0 2023-08-11T15:44:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.492+0000 7f0553fff700 1 -- 172.21.15.32:0/3983994068 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0560021410 con 0x7f056c101aa0 2023-08-11T15:44:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.492+0000 7f0553fff700 1 -- 172.21.15.32:0/3983994068 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560017650 con 0x7f056c101aa0 2023-08-11T15:44:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.492+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c1a0810 con 0x7f056c101aa0 2023-08-11T15:44:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.492+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c1a0be0 con 0x7f056c101aa0 2023-08-11T15:44:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.493+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c107e00 con 0x7f056c101aa0 2023-08-11T15:44:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.497+0000 7f0553fff700 1 -- 172.21.15.32:0/3983994068 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f0560021810 con 0x7f056c101aa0 2023-08-11T15:44:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.497+0000 7f0553fff700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05540a0020 0x7f05540a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.497+0000 7f0553fff700 1 -- 172.21.15.32:0/3983994068 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05600cc190 con 0x7f056c101aa0 2023-08-11T15:44:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.497+0000 7f056affd700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05540a0020 0x7f05540a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.498+0000 7f056affd700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05540a0020 0x7f05540a24e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f055c005950 tx=0x7f055c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:15.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.501+0000 7f0553fff700 1 -- 172.21.15.32:0/3983994068 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f05600963b0 con 0x7f056c101aa0 2023-08-11T15:44:15.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.729+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f056c10c110 con 0x7f056c101aa0 2023-08-11T15:44:15.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.731+0000 7f0553fff700 1 -- 172.21.15.32:0/3983994068 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18396 (secure 0 0 0) 0x7f05600999d0 con 0x7f056c101aa0 2023-08-11T15:44:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:15.733 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:43:32.824336+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:15.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05540a0020 msgr2=0x7f05540a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05540a0020 0x7f05540a24e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f055c005950 tx=0x7f055c009260 comp rx=0 tx=0).stop 2023-08-11T15:44:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 msgr2=0x7f056c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 0x7f056c072d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f056000b680 tx=0x7f0560009580 comp rx=0 tx=0).stop 2023-08-11T15:44:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 shutdown_connections 2023-08-11T15:44:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05540a0020 0x7f05540a24e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056c1008b0 0x7f056c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f056c102440 0x7f056c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 --2- 172.21.15.32:0/3983994068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c101aa0 0x7f056c072d10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 >> 172.21.15.32:0/3983994068 conn(0x7f056c0fbe00 msgr2=0x7f056c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:15.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 shutdown_connections 2023-08-11T15:44:15.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.734+0000 7f05713ff700 1 -- 172.21.15.32:0/3983994068 wait complete. 2023-08-11T15:44:15.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T15:44:15.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:44:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 -- 172.21.15.32:0/2196568126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b00101b20 msgr2=0x7f5b00101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 --2- 172.21.15.32:0/2196568126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b00101b20 0x7f5b00101f80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5af40098f0 tx=0x7f5af4009c00 comp rx=0 tx=0).stop 2023-08-11T15:44:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 -- 172.21.15.32:0/2196568126 shutdown_connections 2023-08-11T15:44:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 --2- 172.21.15.32:0/2196568126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 0x7f5b00100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 --2- 172.21.15.32:0/2196568126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b001024c0 0x7f5b00108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 --2- 172.21.15.32:0/2196568126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b00101b20 0x7f5b00101f80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 -- 172.21.15.32:0/2196568126 >> 172.21.15.32:0/2196568126 conn(0x7f5b000fbea0 msgr2=0x7f5b000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 -- 172.21.15.32:0/2196568126 shutdown_connections 2023-08-11T15:44:15.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.975+0000 7f5b053ff700 1 -- 172.21.15.32:0/2196568126 wait complete. 2023-08-11T15:44:15.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 Processor -- start 2023-08-11T15:44:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 -- start start 2023-08-11T15:44:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 0x7f5b00196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b00101b20 0x7f5b00196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b001024c0 0x7f5b0019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0019e200 con 0x7f5b00100930 2023-08-11T15:44:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0019e340 con 0x7f5b00101b20 2023-08-11T15:44:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.977+0000 7f5b053ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0019e4d0 con 0x7f5b001024c0 2023-08-11T15:44:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 0x7f5b00196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 0x7f5b00196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49912/0 (socket says 172.21.15.32:49912) 2023-08-11T15:44:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 -- 172.21.15.32:0/316390254 learned_addr learned my addr 172.21.15.32:0/316390254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afe7fc700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b00101b20 0x7f5b00196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5aff7fe700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b001024c0 0x7f5b0019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 -- 172.21.15.32:0/316390254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b001024c0 msgr2=0x7f5b0019dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b001024c0 0x7f5b0019dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 -- 172.21.15.32:0/316390254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b00101b20 msgr2=0x7f5b00196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b00101b20 0x7f5b00196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 -- 172.21.15.32:0/316390254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af00095f0 con 0x7f5b00100930 2023-08-11T15:44:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.978+0000 7f5afeffd700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 0x7f5b00196680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5af000ebf0 tx=0x7f5af000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.979+0000 7f5ae7fff700 1 -- 172.21.15.32:0/316390254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0009a10 con 0x7f5b00100930 2023-08-11T15:44:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.979+0000 7f5ae7fff700 1 -- 172.21.15.32:0/316390254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af000cc90 con 0x7f5b00100930 2023-08-11T15:44:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.979+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af4009520 con 0x7f5b00100930 2023-08-11T15:44:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.979+0000 7f5ae7fff700 1 -- 172.21.15.32:0/316390254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0018750 con 0x7f5b00100930 2023-08-11T15:44:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.979+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0019eb80 con 0x7f5b00100930 2023-08-11T15:44:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.980+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b00107f40 con 0x7f5b00100930 2023-08-11T15:44:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.983+0000 7f5ae7fff700 1 -- 172.21.15.32:0/316390254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f5af00188b0 con 0x7f5b00100930 2023-08-11T15:44:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.984+0000 7f5ae7fff700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f5ae809ffd0 0x7f5ae80a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.984+0000 7f5ae7fff700 1 -- 172.21.15.32:0/316390254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5af0014070 con 0x7f5b00100930 2023-08-11T15:44:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.984+0000 7f5afe7fc700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f5ae809ffd0 0x7f5ae80a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.985+0000 7f5afe7fc700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f5ae809ffd0 0x7f5ae80a2490 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5af4005e90 tx=0x7f5af4019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:15.989+0000 7f5ae7fff700 1 -- 172.21.15.32:0/316390254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5af0095560 con 0x7f5b00100930 2023-08-11T15:44:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.203+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b0010c190 con 0x7f5b00100930 2023-08-11T15:44:16.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.203+0000 7f5ae7fff700 1 -- 172.21.15.32:0/316390254 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5af0098b80 con 0x7f5b00100930 2023-08-11T15:44:16.206 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:44:16.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f5ae809ffd0 msgr2=0x7f5ae80a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f5ae809ffd0 0x7f5ae80a2490 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5af4005e90 tx=0x7f5af4019040 comp rx=0 tx=0).stop 2023-08-11T15:44:16.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 msgr2=0x7f5b00196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 0x7f5b00196680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5af000ebf0 tx=0x7f5af000ef00 comp rx=0 tx=0).stop 2023-08-11T15:44:16.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 shutdown_connections 2023-08-11T15:44:16.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f5ae809ffd0 0x7f5ae80a2490 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00100930 0x7f5b00196680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b001024c0 0x7f5b0019dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 --2- 172.21.15.32:0/316390254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b00101b20 0x7f5b00196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 >> 172.21.15.32:0/316390254 conn(0x7f5b000fbea0 msgr2=0x7f5b00105080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:16.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 shutdown_connections 2023-08-11T15:44:16.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.206+0000 7f5b053ff700 1 -- 172.21.15.32:0/316390254 wait complete. 2023-08-11T15:44:16.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-11T15:44:16.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2409161320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:16.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3983994068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:16.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:16.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:44:16.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:16 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.436+0000 7f6982bff700 1 -- 172.21.15.32:0/2469927762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 msgr2=0x7f697c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.436+0000 7f6982bff700 1 --2- 172.21.15.32:0/2469927762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c073800 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6968009990 tx=0x7f6968009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.437+0000 7f6982bff700 1 -- 172.21.15.32:0/2469927762 shutdown_connections 2023-08-11T15:44:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.437+0000 7f6982bff700 1 --2- 172.21.15.32:0/2469927762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f697c10cef0 0x7f697c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.437+0000 7f6982bff700 1 --2- 172.21.15.32:0/2469927762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c074960 0x7f697c072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.437+0000 7f6982bff700 1 --2- 172.21.15.32:0/2469927762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c073800 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.437+0000 7f6982bff700 1 -- 172.21.15.32:0/2469927762 >> 172.21.15.32:0/2469927762 conn(0x7f697c0fbe50 msgr2=0x7f697c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.437+0000 7f6982bff700 1 -- 172.21.15.32:0/2469927762 shutdown_connections 2023-08-11T15:44:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.437+0000 7f6982bff700 1 -- 172.21.15.32:0/2469927762 wait complete. 2023-08-11T15:44:16.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 Processor -- start 2023-08-11T15:44:16.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 -- start start 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f697c074960 0x7f697c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c10cef0 0x7f697c1a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1a47b0 con 0x7f697c073380 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1a4920 con 0x7f697c074960 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f6982bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1a4ae0 con 0x7f697c10cef0 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.438+0000 7f698099b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49928/0 (socket says 172.21.15.32:49928) 2023-08-11T15:44:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 -- 172.21.15.32:0/3921289758 learned_addr learned my addr 172.21.15.32:0/3921289758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f697bfff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f697c074960 0x7f697c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698119c700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c10cef0 0x7f697c1a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 -- 172.21.15.32:0/3921289758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c10cef0 msgr2=0x7f697c1a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c10cef0 0x7f697c1a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 -- 172.21.15.32:0/3921289758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f697c074960 msgr2=0x7f697c19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f697c074960 0x7f697c19d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 -- 172.21.15.32:0/3921289758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69680095f0 con 0x7f697c073380 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698119c700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c10cef0 0x7f697c1a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f698099b700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c19cc70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f697000d950 tx=0x7f697000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f6979ffb700 1 -- 172.21.15.32:0/3921289758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970009940 con 0x7f697c073380 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c1a4dc0 con 0x7f697c073380 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.439+0000 7f6979ffb700 1 -- 172.21.15.32:0/3921289758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6970010460 con 0x7f697c073380 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.440+0000 7f6979ffb700 1 -- 172.21.15.32:0/3921289758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697000f620 con 0x7f697c073380 2023-08-11T15:44:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.440+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1a51f0 con 0x7f697c073380 2023-08-11T15:44:16.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.441+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c102a40 con 0x7f697c073380 2023-08-11T15:44:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.443+0000 7f6979ffb700 1 -- 172.21.15.32:0/3921289758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f69700108d0 con 0x7f697c073380 2023-08-11T15:44:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.444+0000 7f6979ffb700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f696c09ffa0 0x7f696c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.444+0000 7f6979ffb700 1 -- 172.21.15.32:0/3921289758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f69700cafc0 con 0x7f697c073380 2023-08-11T15:44:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.444+0000 7f697bfff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f696c09ffa0 0x7f696c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.445+0000 7f697bfff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f696c09ffa0 0x7f696c0a2460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f696800b5c0 tx=0x7f6968005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:16.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.446+0000 7f6979ffb700 1 -- 172.21.15.32:0/3921289758 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6970095150 con 0x7f697c073380 2023-08-11T15:44:16.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2409161320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:16.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3983994068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:16.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:16.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:44:16.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:16 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2409161320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:16.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3983994068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:16.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:16.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:44:16.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:16 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:16.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.684+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f697c112930 con 0x7f697c073380 2023-08-11T15:44:16.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.684+0000 7f6979ffb700 1 -- 172.21.15.32:0/3921289758 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6970098770 con 0x7f697c073380 2023-08-11T15:44:16.686 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:16.686 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-08-11T15:44:16.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:44:16.686 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-08-11T15:44:16.687 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-08-11T15:44:16.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:44:16.687 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-08-11T15:44:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.686+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f696c09ffa0 msgr2=0x7f696c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.686+0000 7f6982bff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f696c09ffa0 0x7f696c0a2460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f696800b5c0 tx=0x7f6968005fd0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.686+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 msgr2=0x7f697c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.686+0000 7f6982bff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c19cc70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f697000d950 tx=0x7f697000dc60 comp rx=0 tx=0).stop 2023-08-11T15:44:16.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 shutdown_connections 2023-08-11T15:44:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f696c09ffa0 0x7f696c0a2460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c073380 0x7f697c19cc70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c10cef0 0x7f697c1a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 --2- 172.21.15.32:0/3921289758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f697c074960 0x7f697c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 >> 172.21.15.32:0/3921289758 conn(0x7f697c0fbe50 msgr2=0x7f697c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 shutdown_connections 2023-08-11T15:44:16.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.687+0000 7f6982bff700 1 -- 172.21.15.32:0/3921289758 wait complete. 2023-08-11T15:44:16.712 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-11T15:44:16.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-11T15:44:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 -- 172.21.15.32:0/3767912197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c1046f0 msgr2=0x7f987c10afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 --2- 172.21.15.32:0/3767912197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c1046f0 0x7f987c10afa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f98740099c0 tx=0x7f9874009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 -- 172.21.15.32:0/3767912197 shutdown_connections 2023-08-11T15:44:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 --2- 172.21.15.32:0/3767912197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c1046f0 0x7f987c10afa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 --2- 172.21.15.32:0/3767912197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f987c102b60 0x7f987c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 --2- 172.21.15.32:0/3767912197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f987c103d50 0x7f987c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 -- 172.21.15.32:0/3767912197 >> 172.21.15.32:0/3767912197 conn(0x7f987c0fe0d0 msgr2=0x7f987c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 -- 172.21.15.32:0/3767912197 shutdown_connections 2023-08-11T15:44:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.933+0000 7f98817ff700 1 -- 172.21.15.32:0/3767912197 wait complete. 2023-08-11T15:44:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.934+0000 7f98817ff700 1 Processor -- start 2023-08-11T15:44:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.934+0000 7f98817ff700 1 -- start start 2023-08-11T15:44:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f98817ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f987c102b60 0x7f987c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f98817ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c103d50 0x7f987c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f98817ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f987c1046f0 0x7f987c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f98817ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1a0360 con 0x7f987c103d50 2023-08-11T15:44:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f98817ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1a04a0 con 0x7f987c1046f0 2023-08-11T15:44:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f98817ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1a0630 con 0x7f987c102b60 2023-08-11T15:44:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f987a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c103d50 0x7f987c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f987a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c103d50 0x7f987c198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49944/0 (socket says 172.21.15.32:49944) 2023-08-11T15:44:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f987a7fc700 1 -- 172.21.15.32:0/751543146 learned_addr learned my addr 172.21.15.32:0/751543146 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f987affd700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f987c102b60 0x7f987c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f987b7fe700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f987c1046f0 0x7f987c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f987a7fc700 1 -- 172.21.15.32:0/751543146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f987c102b60 msgr2=0x7f987c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.935+0000 7f987a7fc700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f987c102b60 0x7f987c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f987a7fc700 1 -- 172.21.15.32:0/751543146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f987c1046f0 msgr2=0x7f987c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f987a7fc700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f987c1046f0 0x7f987c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f987a7fc700 1 -- 172.21.15.32:0/751543146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864009520 con 0x7f987c103d50 2023-08-11T15:44:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f987a7fc700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c103d50 0x7f987c198d20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f986400eb30 tx=0x7f986400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f9873fff700 1 -- 172.21.15.32:0/751543146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864009880 con 0x7f987c103d50 2023-08-11T15:44:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f9873fff700 1 -- 172.21.15.32:0/751543146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98640044e0 con 0x7f987c103d50 2023-08-11T15:44:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98740095f0 con 0x7f987c103d50 2023-08-11T15:44:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f9873fff700 1 -- 172.21.15.32:0/751543146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864003d80 con 0x7f987c103d50 2023-08-11T15:44:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.936+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c1a0ce0 con 0x7f987c103d50 2023-08-11T15:44:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.938+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c10e3d0 con 0x7f987c103d50 2023-08-11T15:44:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.939+0000 7f9873fff700 1 -- 172.21.15.32:0/751543146 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f9864003f20 con 0x7f987c103d50 2023-08-11T15:44:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.941+0000 7f9873fff700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f986809ffd0 0x7f98680a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.941+0000 7f9873fff700 1 -- 172.21.15.32:0/751543146 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9864014070 con 0x7f987c103d50 2023-08-11T15:44:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.941+0000 7f987affd700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f986809ffd0 0x7f98680a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.942+0000 7f987affd700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f986809ffd0 0x7f98680a2490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f987c103bb0 tx=0x7f986c00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:16.949+0000 7f9873fff700 1 -- 172.21.15.32:0/751543146 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9864095490 con 0x7f987c103d50 2023-08-11T15:44:17.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:17.171+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f987c1a0f60 con 0x7f987c103d50 2023-08-11T15:44:17.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:17 smithi192 ceph-mon[130806]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:17.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:17 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:44:17.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:17 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/316390254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:17.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:17 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3921289758' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-11T15:44:17.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:17 smithi134 ceph-mon[130447]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:17.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:17 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:44:17.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:17 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/316390254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:17.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:17 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3921289758' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-11T15:44:17.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:17 smithi032 ceph-mon[97103]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:17.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:17 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:44:17.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:17 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/316390254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:17.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:17 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3921289758' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-11T15:44:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.089+0000 7f9873fff700 1 -- 172.21.15.32:0/751543146 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9864098ab0 con 0x7f987c103d50 2023-08-11T15:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.093+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f986809ffd0 msgr2=0x7f98680a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.093+0000 7f98817ff700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f986809ffd0 0x7f98680a2490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f987c103bb0 tx=0x7f986c00b040 comp rx=0 tx=0).stop 2023-08-11T15:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.093+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c103d50 msgr2=0x7f987c198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.093+0000 7f98817ff700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c103d50 0x7f987c198d20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f986400eb30 tx=0x7f986400ee40 comp rx=0 tx=0).stop 2023-08-11T15:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.094+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 shutdown_connections 2023-08-11T15:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.094+0000 7f98817ff700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f986809ffd0 0x7f98680a2490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.094+0000 7f98817ff700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f987c1046f0 0x7f987c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.094+0000 7f98817ff700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f987c102b60 0x7f987c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.094+0000 7f98817ff700 1 --2- 172.21.15.32:0/751543146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f987c103d50 0x7f987c198d20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.094+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 >> 172.21.15.32:0/751543146 conn(0x7f987c0fe0d0 msgr2=0x7f987c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.094+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 shutdown_connections 2023-08-11T15:44:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.095+0000 7f98817ff700 1 -- 172.21.15.32:0/751543146 wait complete. 2023-08-11T15:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-08-11T15:44:18.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-08-11T15:44:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.340+0000 7fe5259ff700 1 -- 172.21.15.32:0/2721375651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 msgr2=0x7fe520101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.340+0000 7fe5259ff700 1 --2- 172.21.15.32:0/2721375651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe520101620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe510009990 tx=0x7fe510009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.340+0000 7fe5259ff700 1 -- 172.21.15.32:0/2721375651 shutdown_connections 2023-08-11T15:44:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.340+0000 7fe5259ff700 1 --2- 172.21.15.32:0/2721375651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe520104530 0x7fe5201049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.340+0000 7fe5259ff700 1 --2- 172.21.15.32:0/2721375651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe520101620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.340+0000 7fe5259ff700 1 --2- 172.21.15.32:0/2721375651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520101b60 0x7fe520103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.341+0000 7fe5259ff700 1 -- 172.21.15.32:0/2721375651 >> 172.21.15.32:0/2721375651 conn(0x7fe5200fadf0 msgr2=0x7fe5200fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:18.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.341+0000 7fe5259ff700 1 -- 172.21.15.32:0/2721375651 shutdown_connections 2023-08-11T15:44:18.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.341+0000 7fe5259ff700 1 -- 172.21.15.32:0/2721375651 wait complete. 2023-08-11T15:44:18.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.342+0000 7fe5259ff700 1 Processor -- start 2023-08-11T15:44:18.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.342+0000 7fe5259ff700 1 -- start start 2023-08-11T15:44:18.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe5259ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe52019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe5259ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520101b60 0x7fe52019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe5259ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe520104530 0x7fe5201a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe5259ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201a4790 con 0x7fe520104530 2023-08-11T15:44:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe5259ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201a4900 con 0x7fe520101b60 2023-08-11T15:44:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe5259ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201a4ac0 con 0x7fe5200ff1f0 2023-08-11T15:44:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe52019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:18.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe52019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:55856/0 (socket says 172.21.15.32:55856) 2023-08-11T15:44:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520101b60 0x7fe52019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 -- 172.21.15.32:0/163987554 learned_addr learned my addr 172.21.15.32:0/163987554 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51f7fe700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe520104530 0x7fe5201a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 -- 172.21.15.32:0/163987554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520101b60 msgr2=0x7fe52019d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520101b60 0x7fe52019d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 -- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe520104530 msgr2=0x7fe5201a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe520104530 0x7fe5201a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51effd700 1 -- 172.21.15.32:0/163987554 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100095f0 con 0x7fe5200ff1f0 2023-08-11T15:44:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.343+0000 7fe51e7fc700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520101b60 0x7fe52019d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.344+0000 7fe51effd700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe52019cc50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe510004730 tx=0x7fe510004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.344+0000 7fe5249fd700 1 -- 172.21.15.32:0/163987554 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51001d070 con 0x7fe5200ff1f0 2023-08-11T15:44:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.344+0000 7fe5249fd700 1 -- 172.21.15.32:0/163987554 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe510022470 con 0x7fe5200ff1f0 2023-08-11T15:44:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.344+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5201a4d40 con 0x7fe5200ff1f0 2023-08-11T15:44:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.344+0000 7fe5249fd700 1 -- 172.21.15.32:0/163987554 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510017650 con 0x7fe5200ff1f0 2023-08-11T15:44:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.344+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5201a5160 con 0x7fe5200ff1f0 2023-08-11T15:44:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.345+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5200fc970 con 0x7fe5200ff1f0 2023-08-11T15:44:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.347+0000 7fe5249fd700 1 -- 172.21.15.32:0/163987554 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fe5100225e0 con 0x7fe5200ff1f0 2023-08-11T15:44:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.348+0000 7fe5249fd700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe50809ff50 0x7fe5080a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.349+0000 7fe5249fd700 1 -- 172.21.15.32:0/163987554 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fe5100cc580 con 0x7fe5200ff1f0 2023-08-11T15:44:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.349+0000 7fe51e7fc700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe50809ff50 0x7fe5080a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.349+0000 7fe51e7fc700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe50809ff50 0x7fe5080a2410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe514005950 tx=0x7fe51400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.351+0000 7fe5249fd700 1 -- 172.21.15.32:0/163987554 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe510096630 con 0x7fe5200ff1f0 2023-08-11T15:44:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:18 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/751543146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T15:44:18.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:18 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/751543146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T15:44:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:18.579+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fe520112740 con 0x7fe5200ff1f0 2023-08-11T15:44:18.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:18 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/751543146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T15:44:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.133+0000 7fe5249fd700 1 -- 172.21.15.32:0/163987554 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7fe510099c50 con 0x7fe5200ff1f0 2023-08-11T15:44:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.139+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe50809ff50 msgr2=0x7fe5080a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.139+0000 7fe5259ff700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe50809ff50 0x7fe5080a2410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe514005950 tx=0x7fe51400b410 comp rx=0 tx=0).stop 2023-08-11T15:44:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.139+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 msgr2=0x7fe52019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.139+0000 7fe5259ff700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe52019cc50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe510004730 tx=0x7fe510004760 comp rx=0 tx=0).stop 2023-08-11T15:44:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.143+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 shutdown_connections 2023-08-11T15:44:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.143+0000 7fe5259ff700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe50809ff50 0x7fe5080a2410 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.143+0000 7fe5259ff700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe520104530 0x7fe5201a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.143+0000 7fe5259ff700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5200ff1f0 0x7fe52019cc50 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe510004730 tx=0x7fe510004760 comp rx=0 tx=0).stop 2023-08-11T15:44:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.143+0000 7fe5259ff700 1 --2- 172.21.15.32:0/163987554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520101b60 0x7fe52019d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.143+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 >> 172.21.15.32:0/163987554 conn(0x7fe5200fadf0 msgr2=0x7fe520102870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.149+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 shutdown_connections 2023-08-11T15:44:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.155+0000 7fe5259ff700 1 -- 172.21.15.32:0/163987554 wait complete. 2023-08-11T15:44:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-08-11T15:44:19.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-11T15:44:19.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 -- 172.21.15.32:0/978855790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d0073380 msgr2=0x7f89d0073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:19.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 --2- 172.21.15.32:0/978855790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d0073380 0x7f89d0073800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89c00098f0 tx=0x7f89c0009c00 comp rx=0 tx=0).stop 2023-08-11T15:44:19.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 -- 172.21.15.32:0/978855790 shutdown_connections 2023-08-11T15:44:19.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 --2- 172.21.15.32:0/978855790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d010cee0 0x7f89d010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 --2- 172.21.15.32:0/978855790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d0074960 0x7f89d0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 --2- 172.21.15.32:0/978855790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d0073380 0x7f89d0073800 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 -- 172.21.15.32:0/978855790 >> 172.21.15.32:0/978855790 conn(0x7f89d00fbe60 msgr2=0x7f89d00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:19.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 -- 172.21.15.32:0/978855790 shutdown_connections 2023-08-11T15:44:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.393+0000 7f89d73ff700 1 -- 172.21.15.32:0/978855790 wait complete. 2023-08-11T15:44:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.394+0000 7f89d73ff700 1 Processor -- start 2023-08-11T15:44:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.394+0000 7f89d73ff700 1 -- start start 2023-08-11T15:44:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d73ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d0073380 0x7f89d019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d73ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d0074960 0x7f89d019d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d73ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d010cee0 0x7f89d01a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d73ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01a4830 con 0x7f89d0074960 2023-08-11T15:44:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d73ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01a49a0 con 0x7f89d0073380 2023-08-11T15:44:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d73ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01a4b60 con 0x7f89d010cee0 2023-08-11T15:44:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d499a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d0074960 0x7f89d019d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:19.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d499a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d0074960 0x7f89d019d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49976/0 (socket says 172.21.15.32:49976) 2023-08-11T15:44:19.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d499a700 1 -- 172.21.15.32:0/1685722884 learned_addr learned my addr 172.21.15.32:0/1685722884 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:19.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d519b700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d0073380 0x7f89d019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:19.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.395+0000 7f89d599c700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d010cee0 0x7f89d01a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:19.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d499a700 1 -- 172.21.15.32:0/1685722884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d010cee0 msgr2=0x7f89d01a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d499a700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d010cee0 0x7f89d01a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d499a700 1 -- 172.21.15.32:0/1685722884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d0073380 msgr2=0x7f89d019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d499a700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d0073380 0x7f89d019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d499a700 1 -- 172.21.15.32:0/1685722884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c0009520 con 0x7f89d0074960 2023-08-11T15:44:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d519b700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d0073380 0x7f89d019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:19.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d499a700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d0074960 0x7f89d019d200 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009fd0 tx=0x7f89c000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89c67fc700 1 -- 172.21.15.32:0/1685722884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c001c070 con 0x7f89d0074960 2023-08-11T15:44:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d01a4de0 con 0x7f89d0074960 2023-08-11T15:44:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89c67fc700 1 -- 172.21.15.32:0/1685722884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89c000f7d0 con 0x7f89d0074960 2023-08-11T15:44:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89c67fc700 1 -- 172.21.15.32:0/1685722884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0003a20 con 0x7f89d0074960 2023-08-11T15:44:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.396+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d01a51b0 con 0x7f89d0074960 2023-08-11T15:44:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.397+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d0102a30 con 0x7f89d0074960 2023-08-11T15:44:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.399+0000 7f89c67fc700 1 -- 172.21.15.32:0/1685722884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f89c0020030 con 0x7f89d0074960 2023-08-11T15:44:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.402+0000 7f89c67fc700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89bc09ff50 0x7f89bc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.402+0000 7f89c67fc700 1 -- 172.21.15.32:0/1685722884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f89c00cc370 con 0x7f89d0074960 2023-08-11T15:44:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.402+0000 7f89d519b700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89bc09ff50 0x7f89bc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:19.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.403+0000 7f89d519b700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89bc09ff50 0x7f89bc0a2410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89d019db90 tx=0x7f89cc009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:19.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.405+0000 7f89c67fc700 1 -- 172.21.15.32:0/1685722884 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f89c00962a0 con 0x7f89d0074960 2023-08-11T15:44:19.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:19 smithi134 ceph-mon[130447]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:19.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:19 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/751543146' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-11T15:44:19.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:19 smithi134 ceph-mon[130447]: osdmap e81: 12 total, 12 up, 12 in 2023-08-11T15:44:19.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:19 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/163987554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T15:44:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:19 smithi032 ceph-mon[97103]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:19 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/751543146' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-11T15:44:19.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:19 smithi032 ceph-mon[97103]: osdmap e81: 12 total, 12 up, 12 in 2023-08-11T15:44:19.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:19 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/163987554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T15:44:19.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:19 smithi192 ceph-mon[130806]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:19.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:19 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/751543146' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-11T15:44:19.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:19 smithi192 ceph-mon[130806]: osdmap e81: 12 total, 12 up, 12 in 2023-08-11T15:44:19.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:19 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/163987554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T15:44:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:19.705+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f89d0112840 con 0x7f89d0074960 2023-08-11T15:44:20.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.163+0000 7f89c67fc700 1 -- 172.21.15.32:0/1685722884 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f89c00998c0 con 0x7f89d0074960 2023-08-11T15:44:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89bc09ff50 msgr2=0x7f89bc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89bc09ff50 0x7f89bc0a2410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89d019db90 tx=0x7f89cc009310 comp rx=0 tx=0).stop 2023-08-11T15:44:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d0074960 msgr2=0x7f89d019d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d0074960 0x7f89d019d200 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009fd0 tx=0x7f89c000b790 comp rx=0 tx=0).stop 2023-08-11T15:44:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 shutdown_connections 2023-08-11T15:44:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f89bc09ff50 0x7f89bc0a2410 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d010cee0 0x7f89d01a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d0073380 0x7f89d019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 --2- 172.21.15.32:0/1685722884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d0074960 0x7f89d019d200 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 >> 172.21.15.32:0/1685722884 conn(0x7f89d00fbe60 msgr2=0x7f89d00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 shutdown_connections 2023-08-11T15:44:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.166+0000 7f89d73ff700 1 -- 172.21.15.32:0/1685722884 wait complete. 2023-08-11T15:44:20.171 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2023-08-11T15:44:20.189 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-08-11T15:44:20.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-08-11T15:44:20.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 -- 172.21.15.32:0/1363472426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 msgr2=0x7f0960100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:20.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 --2- 172.21.15.32:0/1363472426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960100d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09500099c0 tx=0x7f0950009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 -- 172.21.15.32:0/1363472426 shutdown_connections 2023-08-11T15:44:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 --2- 172.21.15.32:0/1363472426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960100d30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 --2- 172.21.15.32:0/1363472426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09601024a0 0x7f0960108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 --2- 172.21.15.32:0/1363472426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0960101b00 0x7f0960101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 -- 172.21.15.32:0/1363472426 >> 172.21.15.32:0/1363472426 conn(0x7f09600fbe60 msgr2=0x7f09600fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 -- 172.21.15.32:0/1363472426 shutdown_connections 2023-08-11T15:44:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.398+0000 7f0964dff700 1 -- 172.21.15.32:0/1363472426 wait complete. 2023-08-11T15:44:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.399+0000 7f0964dff700 1 Processor -- start 2023-08-11T15:44:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.399+0000 7f0964dff700 1 -- start start 2023-08-11T15:44:20.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f0964dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f0964dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0960101b00 0x7f0960196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f0964dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09601024a0 0x7f096019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f0964dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096019e1d0 con 0x7f0960100910 2023-08-11T15:44:20.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f0964dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096019e310 con 0x7f09601024a0 2023-08-11T15:44:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f0964dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096019e4a0 con 0x7f0960101b00 2023-08-11T15:44:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49982/0 (socket says 172.21.15.32:49982) 2023-08-11T15:44:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 -- 172.21.15.32:0/2519640613 learned_addr learned my addr 172.21.15.32:0/2519640613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:20.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095ed9d700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09601024a0 0x7f096019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:20.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f0957fff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0960101b00 0x7f0960196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:20.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 -- 172.21.15.32:0/2519640613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0960101b00 msgr2=0x7f0960196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:20.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0960101b00 0x7f0960196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 -- 172.21.15.32:0/2519640613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09601024a0 msgr2=0x7f096019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09601024a0 0x7f096019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.400+0000 7f095e59c700 1 -- 172.21.15.32:0/2519640613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500095f0 con 0x7f0960100910 2023-08-11T15:44:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.401+0000 7f095e59c700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960196650 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0950009f50 tx=0x7f0950005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.401+0000 7f09577fe700 1 -- 172.21.15.32:0/2519640613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095001d070 con 0x7f0960100910 2023-08-11T15:44:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.401+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096019e720 con 0x7f0960100910 2023-08-11T15:44:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.401+0000 7f09577fe700 1 -- 172.21.15.32:0/2519640613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0950003f90 con 0x7f0960100910 2023-08-11T15:44:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.401+0000 7f09577fe700 1 -- 172.21.15.32:0/2519640613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950022d00 con 0x7f0960100910 2023-08-11T15:44:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.401+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096019eb50 con 0x7f0960100910 2023-08-11T15:44:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.402+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0960107890 con 0x7f0960100910 2023-08-11T15:44:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.404+0000 7f09577fe700 1 -- 172.21.15.32:0/2519640613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f0950022470 con 0x7f0960100910 2023-08-11T15:44:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.407+0000 7f09577fe700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f094c09ff50 0x7f094c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:20.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.407+0000 7f09577fe700 1 -- 172.21.15.32:0/2519640613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09500cc350 con 0x7f0960100910 2023-08-11T15:44:20.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.407+0000 7f0957fff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f094c09ff50 0x7f094c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:20.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.407+0000 7f0957fff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f094c09ff50 0x7f094c0a2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0948007c60 tx=0x7f09480073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:20.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.411+0000 7f09577fe700 1 -- 172.21.15.32:0/2519640613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0950096220 con 0x7f0960100910 2023-08-11T15:44:20.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T15:44:20.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-11T15:44:20.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: osdmap e82: 12 total, 12 up, 12 in 2023-08-11T15:44:20.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1685722884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T15:44:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-11T15:44:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: osdmap e83: 12 total, 12 up, 12 in 2023-08-11T15:44:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:20.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:44:20.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T15:44:20.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-11T15:44:20.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: osdmap e82: 12 total, 12 up, 12 in 2023-08-11T15:44:20.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.562 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1685722884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T15:44:20.562 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-11T15:44:20.562 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: osdmap e83: 12 total, 12 up, 12 in 2023-08-11T15:44:20.562 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.562 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.562 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:20.562 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:44:20.564 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T15:44:20.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-11T15:44:20.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: osdmap e82: 12 total, 12 up, 12 in 2023-08-11T15:44:20.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1685722884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T15:44:20.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-11T15:44:20.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: osdmap e83: 12 total, 12 up, 12 in 2023-08-11T15:44:20.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:44:20.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:44:20.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:44:20.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.636+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f096010c170 con 0x7f0960100910 2023-08-11T15:44:20.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.852+0000 7f09577fe700 1 -- 172.21.15.32:0/2519640613 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f0950022e60 con 0x7f0960100910 2023-08-11T15:44:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f094c09ff50 msgr2=0x7f094c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f094c09ff50 0x7f094c0a2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0948007c60 tx=0x7f09480073d0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 msgr2=0x7f0960196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960196650 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0950009f50 tx=0x7f0950005740 comp rx=0 tx=0).stop 2023-08-11T15:44:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 shutdown_connections 2023-08-11T15:44:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f094c09ff50 0x7f094c0a2410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0960100910 0x7f0960196650 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09601024a0 0x7f096019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 --2- 172.21.15.32:0/2519640613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0960101b00 0x7f0960196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 >> 172.21.15.32:0/2519640613 conn(0x7f09600fbe60 msgr2=0x7f09600fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 shutdown_connections 2023-08-11T15:44:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:20.855+0000 7f0964dff700 1 -- 172.21.15.32:0/2519640613 wait complete. 2023-08-11T15:44:20.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-08-11T15:44:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 -- 172.21.15.32:0/314204099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 msgr2=0x7f2efc073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 --2- 172.21.15.32:0/314204099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc073800 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009990 tx=0x7f2eec009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 -- 172.21.15.32:0/314204099 shutdown_connections 2023-08-11T15:44:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 --2- 172.21.15.32:0/314204099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc10cef0 0x7f2efc10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 --2- 172.21.15.32:0/314204099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc074960 0x7f2efc072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 --2- 172.21.15.32:0/314204099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc073800 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 -- 172.21.15.32:0/314204099 >> 172.21.15.32:0/314204099 conn(0x7f2efc0fbe50 msgr2=0x7f2efc0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 -- 172.21.15.32:0/314204099 shutdown_connections 2023-08-11T15:44:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.087+0000 7f2f043ff700 1 -- 172.21.15.32:0/314204099 wait complete. 2023-08-11T15:44:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.088+0000 7f2f043ff700 1 Processor -- start 2023-08-11T15:44:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.088+0000 7f2f043ff700 1 -- start start 2023-08-11T15:44:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f043ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f043ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc074960 0x7f2efc19d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f043ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc10cef0 0x7f2efc1a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f043ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc1a4720 con 0x7f2efc073380 2023-08-11T15:44:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f043ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc1a4860 con 0x7f2efc074960 2023-08-11T15:44:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f043ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc1a49f0 con 0x7f2efc10cef0 2023-08-11T15:44:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49996/0 (socket says 172.21.15.32:49996) 2023-08-11T15:44:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 -- 172.21.15.32:0/4184768042 learned_addr learned my addr 172.21.15.32:0/4184768042 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0199a700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc074960 0x7f2efc19d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0299c700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc10cef0 0x7f2efc1a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 -- 172.21.15.32:0/4184768042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc10cef0 msgr2=0x7f2efc1a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc10cef0 0x7f2efc1a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 -- 172.21.15.32:0/4184768042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc074960 msgr2=0x7f2efc19d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc074960 0x7f2efc19d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.089+0000 7f2f0219b700 1 -- 172.21.15.32:0/4184768042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0095f0 con 0x7f2efc073380 2023-08-11T15:44:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.090+0000 7f2f0219b700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc19cba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9a0 tx=0x7f2ef800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.090+0000 7f2ef37fe700 1 -- 172.21.15.32:0/4184768042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8009790 con 0x7f2efc073380 2023-08-11T15:44:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.090+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc1a4cd0 con 0x7f2efc073380 2023-08-11T15:44:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.090+0000 7f2ef37fe700 1 -- 172.21.15.32:0/4184768042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef800cbd0 con 0x7f2efc073380 2023-08-11T15:44:21.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.090+0000 7f2ef37fe700 1 -- 172.21.15.32:0/4184768042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8018680 con 0x7f2efc073380 2023-08-11T15:44:21.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.090+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc1a5180 con 0x7f2efc073380 2023-08-11T15:44:21.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.091+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc102a40 con 0x7f2efc073380 2023-08-11T15:44:21.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.093+0000 7f2ef37fe700 1 -- 172.21.15.32:0/4184768042 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f2ef800cd40 con 0x7f2efc073380 2023-08-11T15:44:21.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.095+0000 7f2ef37fe700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2ee809fff0 0x7f2ee80a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:21.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.096+0000 7f2ef37fe700 1 -- 172.21.15.32:0/4184768042 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f2ef8014070 con 0x7f2efc073380 2023-08-11T15:44:21.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.096+0000 7f2f0199a700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2ee809fff0 0x7f2ee80a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:21.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.096+0000 7f2f0199a700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2ee809fff0 0x7f2ee80a24b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b2e0 tx=0x7f2eec005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.099+0000 7f2ef37fe700 1 -- 172.21.15.32:0/4184768042 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2ef8051c30 con 0x7f2efc073380 2023-08-11T15:44:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:21.314+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2efc112760 con 0x7f2efc073380 2023-08-11T15:44:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:21 smithi134 ceph-mon[130447]: pgmap v266: 97 pgs: 11 creating+peering, 24 active+clean, 62 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:21 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1685722884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-11T15:44:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:21 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-11T15:44:21.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:21 smithi134 ceph-mon[130447]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-11T15:44:21.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:21 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2519640613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T15:44:21.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:21 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2519640613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-11T15:44:21.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:21 smithi134 ceph-mon[130447]: osdmap e84: 12 total, 12 up, 12 in 2023-08-11T15:44:21.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:21 smithi032 ceph-mon[97103]: pgmap v266: 97 pgs: 11 creating+peering, 24 active+clean, 62 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:21.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:21 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1685722884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-11T15:44:21.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:21 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-11T15:44:21.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:21 smithi032 ceph-mon[97103]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-11T15:44:21.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:21 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2519640613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T15:44:21.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:21 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2519640613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-11T15:44:21.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:21 smithi032 ceph-mon[97103]: osdmap e84: 12 total, 12 up, 12 in 2023-08-11T15:44:21.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:21 smithi192 ceph-mon[130806]: pgmap v266: 97 pgs: 11 creating+peering, 24 active+clean, 62 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:44:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:21 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1685722884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-11T15:44:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:21 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-11T15:44:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:21 smithi192 ceph-mon[130806]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-11T15:44:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:21 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2519640613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T15:44:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:21 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2519640613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-11T15:44:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:21 smithi192 ceph-mon[130806]: osdmap e84: 12 total, 12 up, 12 in 2023-08-11T15:44:22.971 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:44:22 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b[155523]: 2023-08-11T15:44:22.828+0000 7f1c19333700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-11T15:44:22.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:22 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] up:active 2023-08-11T15:44:22.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:22 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:22.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:22 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:22.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:22 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4184768042' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T15:44:23.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:22 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] up:active 2023-08-11T15:44:23.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:22 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:23.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:22 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:23.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:22 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4184768042' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T15:44:23.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:22 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] up:active 2023-08-11T15:44:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:22 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:22 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:23.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:22 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4184768042' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T15:44:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.177+0000 7f2ef37fe700 1 -- 172.21.15.32:0/4184768042 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f2ef8095200 con 0x7f2efc073380 2023-08-11T15:44:23.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.184+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2ee809fff0 msgr2=0x7f2ee80a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:23.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.184+0000 7f2f043ff700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2ee809fff0 0x7f2ee80a24b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b2e0 tx=0x7f2eec005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.184+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 msgr2=0x7f2efc19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:23.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.184+0000 7f2f043ff700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc19cba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9a0 tx=0x7f2ef800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.185+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 shutdown_connections 2023-08-11T15:44:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.185+0000 7f2f043ff700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2ee809fff0 0x7f2ee80a24b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.185+0000 7f2f043ff700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2efc073380 0x7f2efc19cba0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.185+0000 7f2f043ff700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc10cef0 0x7f2efc1a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.185+0000 7f2f043ff700 1 --2- 172.21.15.32:0/4184768042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc074960 0x7f2efc19d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.185+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 >> 172.21.15.32:0/4184768042 conn(0x7f2efc0fbe50 msgr2=0x7f2efc0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.186+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 shutdown_connections 2023-08-11T15:44:23.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.186+0000 7f2f043ff700 1 -- 172.21.15.32:0/4184768042 wait complete. 2023-08-11T15:44:23.189 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data_ec' created 2023-08-11T15:44:23.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-08-11T15:44:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 -- 172.21.15.32:0/2070987546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 msgr2=0x7f423c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 --2- 172.21.15.32:0/2070987546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c100d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f422c0099c0 tx=0x7f422c009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 -- 172.21.15.32:0/2070987546 shutdown_connections 2023-08-11T15:44:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 --2- 172.21.15.32:0/2070987546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c100d50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 --2- 172.21.15.32:0/2070987546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f423c1024c0 0x7f423c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 --2- 172.21.15.32:0/2070987546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f423c101b20 0x7f423c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 -- 172.21.15.32:0/2070987546 >> 172.21.15.32:0/2070987546 conn(0x7f423c0fbea0 msgr2=0x7f423c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 -- 172.21.15.32:0/2070987546 shutdown_connections 2023-08-11T15:44:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.417+0000 7f4241bff700 1 -- 172.21.15.32:0/2070987546 wait complete. 2023-08-11T15:44:23.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.418+0000 7f4241bff700 1 Processor -- start 2023-08-11T15:44:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.418+0000 7f4241bff700 1 -- start start 2023-08-11T15:44:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.418+0000 7f4241bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.418+0000 7f4241bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f423c101b20 0x7f423c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f4241bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f423c1024c0 0x7f423c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f4241bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0739f0 con 0x7f423c100930 2023-08-11T15:44:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f4241bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c073b60 con 0x7f423c1024c0 2023-08-11T15:44:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f4241bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c1a0730 con 0x7f423c101b20 2023-08-11T15:44:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50016/0 (socket says 172.21.15.32:50016) 2023-08-11T15:44:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 -- 172.21.15.32:0/3289715718 learned_addr learned my addr 172.21.15.32:0/3289715718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423affd700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f423c101b20 0x7f423c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423bfff700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f423c1024c0 0x7f423c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 -- 172.21.15.32:0/3289715718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f423c101b20 msgr2=0x7f423c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f423c101b20 0x7f423c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 -- 172.21.15.32:0/3289715718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f423c1024c0 msgr2=0x7f423c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f423c1024c0 0x7f423c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 -- 172.21.15.32:0/3289715718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0095f0 con 0x7f423c100930 2023-08-11T15:44:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.419+0000 7f423b7fe700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c0746d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f422c009fd0 tx=0x7f422c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.420+0000 7f4238ff9700 1 -- 172.21.15.32:0/3289715718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c01d070 con 0x7f423c100930 2023-08-11T15:44:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.420+0000 7f4238ff9700 1 -- 172.21.15.32:0/3289715718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422c005510 con 0x7f423c100930 2023-08-11T15:44:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.420+0000 7f4238ff9700 1 -- 172.21.15.32:0/3289715718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c004b00 con 0x7f423c100930 2023-08-11T15:44:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.420+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c1a09b0 con 0x7f423c100930 2023-08-11T15:44:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.420+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c1a0dd0 con 0x7f423c100930 2023-08-11T15:44:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.421+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c107ec0 con 0x7f423c100930 2023-08-11T15:44:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.423+0000 7f4238ff9700 1 -- 172.21.15.32:0/3289715718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f422c00fb80 con 0x7f423c100930 2023-08-11T15:44:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.424+0000 7f4238ff9700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f422409ffd0 0x7f42240a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.425+0000 7f4238ff9700 1 -- 172.21.15.32:0/3289715718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f422c0cc640 con 0x7f423c100930 2023-08-11T15:44:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.425+0000 7f423affd700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f422409ffd0 0x7f42240a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.425+0000 7f423affd700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f422409ffd0 0x7f42240a2490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4230009c60 tx=0x7f4230009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:23.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.427+0000 7f4238ff9700 1 -- 172.21.15.32:0/3289715718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f422c096230 con 0x7f423c100930 2023-08-11T15:44:23.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:23.634+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f423c10c3e0 con 0x7f423c100930 2023-08-11T15:44:23.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:23 smithi134 ceph-mon[130447]: pgmap v268: 97 pgs: 11 creating+peering, 52 active+clean, 34 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-08-11T15:44:23.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:23 smithi134 ceph-mon[130447]: osdmap e85: 12 total, 12 up, 12 in 2023-08-11T15:44:23.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:23 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4184768042' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-11T15:44:23.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:23 smithi134 ceph-mon[130447]: osdmap e86: 12 total, 12 up, 12 in 2023-08-11T15:44:23.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:23 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3289715718' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-11T15:44:24.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:23 smithi032 ceph-mon[97103]: pgmap v268: 97 pgs: 11 creating+peering, 52 active+clean, 34 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-08-11T15:44:24.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:23 smithi032 ceph-mon[97103]: osdmap e85: 12 total, 12 up, 12 in 2023-08-11T15:44:24.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:23 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4184768042' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-11T15:44:24.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:23 smithi032 ceph-mon[97103]: osdmap e86: 12 total, 12 up, 12 in 2023-08-11T15:44:24.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:23 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3289715718' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-11T15:44:24.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:23 smithi192 ceph-mon[130806]: pgmap v268: 97 pgs: 11 creating+peering, 52 active+clean, 34 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-08-11T15:44:24.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:23 smithi192 ceph-mon[130806]: osdmap e85: 12 total, 12 up, 12 in 2023-08-11T15:44:24.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:23 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4184768042' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-11T15:44:24.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:23 smithi192 ceph-mon[130806]: osdmap e86: 12 total, 12 up, 12 in 2023-08-11T15:44:24.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:23 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3289715718' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-11T15:44:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.179+0000 7f4238ff9700 1 -- 172.21.15.32:0/3289715718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f422c099850 con 0x7f423c100930 2023-08-11T15:44:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.188+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f422409ffd0 msgr2=0x7f42240a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.188+0000 7f4241bff700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f422409ffd0 0x7f42240a2490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4230009c60 tx=0x7f4230009260 comp rx=0 tx=0).stop 2023-08-11T15:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.188+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 msgr2=0x7f423c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.188+0000 7f4241bff700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c0746d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f422c009fd0 tx=0x7f422c00f690 comp rx=0 tx=0).stop 2023-08-11T15:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.189+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 shutdown_connections 2023-08-11T15:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.189+0000 7f4241bff700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f422409ffd0 0x7f42240a2490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.189+0000 7f4241bff700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423c100930 0x7f423c0746d0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f422c009fd0 tx=0x7f422c00f690 comp rx=0 tx=0).stop 2023-08-11T15:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.189+0000 7f4241bff700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f423c1024c0 0x7f423c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.189+0000 7f4241bff700 1 --2- 172.21.15.32:0/3289715718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f423c101b20 0x7f423c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.189+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 >> 172.21.15.32:0/3289715718 conn(0x7f423c0fbea0 msgr2=0x7f423c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:24.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.195+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 shutdown_connections 2023-08-11T15:44:24.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.196+0000 7f4241bff700 1 -- 172.21.15.32:0/3289715718 wait complete. 2023-08-11T15:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 4 allow_ec_overwrites to true 2023-08-11T15:44:24.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-08-11T15:44:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 -- 172.21.15.32:0/1670843772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4101b00 msgr2=0x7f7df4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 --2- 172.21.15.32:0/1670843772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4101b00 0x7f7df4101f60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7de80099c0 tx=0x7f7de8009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 -- 172.21.15.32:0/1670843772 shutdown_connections 2023-08-11T15:44:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 --2- 172.21.15.32:0/1670843772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df41024a0 0x7f7df4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 --2- 172.21.15.32:0/1670843772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df4100910 0x7f7df4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 --2- 172.21.15.32:0/1670843772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4101b00 0x7f7df4101f60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 -- 172.21.15.32:0/1670843772 >> 172.21.15.32:0/1670843772 conn(0x7f7df40fbe60 msgr2=0x7f7df40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 -- 172.21.15.32:0/1670843772 shutdown_connections 2023-08-11T15:44:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.430+0000 7f7df99ff700 1 -- 172.21.15.32:0/1670843772 wait complete. 2023-08-11T15:44:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.431+0000 7f7df99ff700 1 Processor -- start 2023-08-11T15:44:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.431+0000 7f7df99ff700 1 -- start start 2023-08-11T15:44:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df99ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4100910 0x7f7df4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df99ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df4101b00 0x7f7df4198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df99ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df41024a0 0x7f7df419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df99ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41a0400 con 0x7f7df4100910 2023-08-11T15:44:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df99ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41a0540 con 0x7f7df41024a0 2023-08-11T15:44:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df99ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41a06d0 con 0x7f7df4101b00 2023-08-11T15:44:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4100910 0x7f7df4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4100910 0x7f7df4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50028/0 (socket says 172.21.15.32:50028) 2023-08-11T15:44:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 -- 172.21.15.32:0/313516958 learned_addr learned my addr 172.21.15.32:0/313516958 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df27fc700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df4101b00 0x7f7df4198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df37fe700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df41024a0 0x7f7df419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 -- 172.21.15.32:0/313516958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df4101b00 msgr2=0x7f7df4198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df4101b00 0x7f7df4198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 -- 172.21.15.32:0/313516958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df41024a0 msgr2=0x7f7df419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df41024a0 0x7f7df419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.432+0000 7f7df2ffd700 1 -- 172.21.15.32:0/313516958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80095f0 con 0x7f7df4100910 2023-08-11T15:44:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.433+0000 7f7df2ffd700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4100910 0x7f7df4198880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7de400d980 tx=0x7f7de400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.433+0000 7f7df89fd700 1 -- 172.21.15.32:0/313516958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4004020 con 0x7f7df4100910 2023-08-11T15:44:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.433+0000 7f7df89fd700 1 -- 172.21.15.32:0/313516958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7de400f690 con 0x7f7df4100910 2023-08-11T15:44:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.433+0000 7f7df89fd700 1 -- 172.21.15.32:0/313516958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4003a60 con 0x7f7df4100910 2023-08-11T15:44:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.433+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df41a09b0 con 0x7f7df4100910 2023-08-11T15:44:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.433+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df41a0e70 con 0x7f7df4100910 2023-08-11T15:44:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.434+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df410c170 con 0x7f7df4100910 2023-08-11T15:44:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.436+0000 7f7df89fd700 1 -- 172.21.15.32:0/313516958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f7de400fb00 con 0x7f7df4100910 2023-08-11T15:44:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.437+0000 7f7df89fd700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f7ddc09ffd0 0x7f7ddc0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.438+0000 7f7df89fd700 1 -- 172.21.15.32:0/313516958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f7de40caf60 con 0x7f7df4100910 2023-08-11T15:44:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.438+0000 7f7df27fc700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f7ddc09ffd0 0x7f7ddc0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.438+0000 7f7df27fc700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f7ddc09ffd0 0x7f7ddc0a2490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b580 tx=0x7f7de8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:24.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.441+0000 7f7df89fd700 1 -- 172.21.15.32:0/313516958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7de4094ac0 con 0x7f7df4100910 2023-08-11T15:44:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:24.700+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f7df410c380 con 0x7f7df4100910 2023-08-11T15:44:25.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:25 smithi134 ceph-mon[130447]: pgmap v271: 129 pgs: 32 unknown, 8 creating+peering, 89 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 5 op/s 2023-08-11T15:44:25.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:25 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3289715718' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-11T15:44:25.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:25 smithi134 ceph-mon[130447]: osdmap e87: 12 total, 12 up, 12 in 2023-08-11T15:44:25.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:25 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/313516958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-11T15:44:25.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:25 smithi032 ceph-mon[97103]: pgmap v271: 129 pgs: 32 unknown, 8 creating+peering, 89 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 5 op/s 2023-08-11T15:44:25.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:25 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3289715718' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-11T15:44:25.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:25 smithi032 ceph-mon[97103]: osdmap e87: 12 total, 12 up, 12 in 2023-08-11T15:44:25.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:25 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/313516958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-11T15:44:25.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:25 smithi192 ceph-mon[130806]: pgmap v271: 129 pgs: 32 unknown, 8 creating+peering, 89 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 5 op/s 2023-08-11T15:44:25.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:25 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3289715718' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-11T15:44:25.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:25 smithi192 ceph-mon[130806]: osdmap e87: 12 total, 12 up, 12 in 2023-08-11T15:44:25.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:25 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/313516958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-11T15:44:25.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.692+0000 7f7df89fd700 1 -- 172.21.15.32:0/313516958 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f7de40980e0 con 0x7f7df4100910 2023-08-11T15:44:25.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f7ddc09ffd0 msgr2=0x7f7ddc0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:25.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f7ddc09ffd0 0x7f7ddc0a2490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b580 tx=0x7f7de8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4100910 msgr2=0x7f7df4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:25.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4100910 0x7f7df4198880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7de400d980 tx=0x7f7de400dc90 comp rx=0 tx=0).stop 2023-08-11T15:44:25.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 shutdown_connections 2023-08-11T15:44:25.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f7ddc09ffd0 0x7f7ddc0a2490 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4100910 0x7f7df4198880 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7df41024a0 0x7f7df419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 --2- 172.21.15.32:0/313516958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df4101b00 0x7f7df4198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 >> 172.21.15.32:0/313516958 conn(0x7f7df40fbe60 msgr2=0x7f7df40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:25.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 shutdown_connections 2023-08-11T15:44:25.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.696+0000 7f7df99ff700 1 -- 172.21.15.32:0/313516958 wait complete. 2023-08-11T15:44:25.701 INFO:teuthology.orchestra.run.smithi032.stderr:added data pool 4 to fsmap 2023-08-11T15:44:25.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:25.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.926+0000 7fa9d0bff700 1 -- 172.21.15.32:0/3694690568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 msgr2=0x7fa9cc073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:25.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.926+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/3694690568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 0x7fa9cc073800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40098f0 tx=0x7fa9b4009c00 comp rx=0 tx=0).stop 2023-08-11T15:44:25.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.926+0000 7fa9d0bff700 1 -- 172.21.15.32:0/3694690568 shutdown_connections 2023-08-11T15:44:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.926+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/3694690568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc074960 0x7fa9cc072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.926+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/3694690568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9cc1087f0 0x7fa9cc10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.926+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/3694690568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 0x7fa9cc073800 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.926+0000 7fa9d0bff700 1 -- 172.21.15.32:0/3694690568 >> 172.21.15.32:0/3694690568 conn(0x7fa9cc0fbe60 msgr2=0x7fa9cc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.927+0000 7fa9d0bff700 1 -- 172.21.15.32:0/3694690568 shutdown_connections 2023-08-11T15:44:25.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.927+0000 7fa9d0bff700 1 -- 172.21.15.32:0/3694690568 wait complete. 2023-08-11T15:44:25.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.927+0000 7fa9d0bff700 1 Processor -- start 2023-08-11T15:44:25.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9d0bff700 1 -- start start 2023-08-11T15:44:25.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9d0bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 0x7fa9cc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:25.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9d0bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9cc074960 0x7fa9cc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9d0bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc1087f0 0x7fa9cc19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9d0bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1a03f0 con 0x7fa9cc1087f0 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9d0bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1a0530 con 0x7fa9cc074960 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9d0bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1a06c0 con 0x7fa9cc073380 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc1087f0 0x7fa9cc19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc1087f0 0x7fa9cc19fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34864/0 (socket says 172.21.15.32:34864) 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9cad9d700 1 -- 172.21.15.32:0/2098792582 learned_addr learned my addr 172.21.15.32:0/2098792582 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9ca59c700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 0x7fa9cc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.928+0000 7fa9c3fff700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9cc074960 0x7fa9cc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9cad9d700 1 -- 172.21.15.32:0/2098792582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 msgr2=0x7fa9cc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9cad9d700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 0x7fa9cc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9cad9d700 1 -- 172.21.15.32:0/2098792582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9cc074960 msgr2=0x7fa9cc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9cad9d700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9cc074960 0x7fa9cc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9cad9d700 1 -- 172.21.15.32:0/2098792582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0095f0 con 0x7fa9cc1087f0 2023-08-11T15:44:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9cad9d700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc1087f0 0x7fa9cc19fe20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400ef30 tx=0x7fa9c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9c37fe700 1 -- 172.21.15.32:0/2098792582 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4018070 con 0x7fa9cc1087f0 2023-08-11T15:44:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9c37fe700 1 -- 172.21.15.32:0/2098792582 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9c400f040 con 0x7fa9cc1087f0 2023-08-11T15:44:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b4009520 con 0x7fa9cc1087f0 2023-08-11T15:44:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9c37fe700 1 -- 172.21.15.32:0/2098792582 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400cd00 con 0x7fa9cc1087f0 2023-08-11T15:44:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.929+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc1a0cc0 con 0x7fa9cc1087f0 2023-08-11T15:44:25.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.930+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc106ea0 con 0x7fa9cc1087f0 2023-08-11T15:44:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.932+0000 7fa9c37fe700 1 -- 172.21.15.32:0/2098792582 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fa9c4003a00 con 0x7fa9cc1087f0 2023-08-11T15:44:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.934+0000 7fa9c37fe700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa9b80a0020 0x7fa9b80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.934+0000 7fa9c37fe700 1 -- 172.21.15.32:0/2098792582 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa9c40caf70 con 0x7fa9cc1087f0 2023-08-11T15:44:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.934+0000 7fa9ca59c700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa9b80a0020 0x7fa9b80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.935+0000 7fa9ca59c700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa9b80a0020 0x7fa9b80a24e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc005fd0 tx=0x7fa9bc009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:25.937+0000 7fa9c37fe700 1 -- 172.21.15.32:0/2098792582 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa9c4094b40 con 0x7fa9cc1087f0 2023-08-11T15:44:26.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.197+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9cc10e370 con 0x7fa9cc1087f0 2023-08-11T15:44:26.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.198+0000 7fa9c37fe700 1 -- 172.21.15.32:0/2098792582 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11264 (secure 0 0 0) 0x7fa9c4098160 con 0x7fa9cc1087f0 2023-08-11T15:44:26.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:26.200 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:25.687892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-11T15:44:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.200+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa9b80a0020 msgr2=0x7fa9b80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.200+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa9b80a0020 0x7fa9b80a24e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc005fd0 tx=0x7fa9bc009310 comp rx=0 tx=0).stop 2023-08-11T15:44:26.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.200+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc1087f0 msgr2=0x7fa9cc19fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.200+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc1087f0 0x7fa9cc19fe20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400ef30 tx=0x7fa9c400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 shutdown_connections 2023-08-11T15:44:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa9b80a0020 0x7fa9b80a24e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9cc1087f0 0x7fa9cc19fe20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc073380 0x7fa9cc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 --2- 172.21.15.32:0/2098792582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9cc074960 0x7fa9cc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 >> 172.21.15.32:0/2098792582 conn(0x7fa9cc0fbe60 msgr2=0x7fa9cc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 shutdown_connections 2023-08-11T15:44:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.201+0000 7fa9d0bff700 1 -- 172.21.15.32:0/2098792582 wait complete. 2023-08-11T15:44:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2023-08-11T15:44:26.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 -- 172.21.15.32:0/1241208929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 msgr2=0x7fa168101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/1241208929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168101f80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0099c0 tx=0x7fa15c009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 -- 172.21.15.32:0/1241208929 shutdown_connections 2023-08-11T15:44:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/1241208929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1681024c0 0x7fa168108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/1241208929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa168100930 0x7fa168100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/1241208929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168101f80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 -- 172.21.15.32:0/1241208929 >> 172.21.15.32:0/1241208929 conn(0x7fa1680fbe60 msgr2=0x7fa1680fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 -- 172.21.15.32:0/1241208929 shutdown_connections 2023-08-11T15:44:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.424+0000 7fa16d3ff700 1 -- 172.21.15.32:0/1241208929 wait complete. 2023-08-11T15:44:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.425+0000 7fa16d3ff700 1 Processor -- start 2023-08-11T15:44:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.425+0000 7fa16d3ff700 1 -- start start 2023-08-11T15:44:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa16d3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa168100930 0x7fa1681987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa16d3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa16d3ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1681024c0 0x7fa16819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa16d3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1681a0370 con 0x7fa168101b20 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa16d3ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1681a04b0 con 0x7fa1681024c0 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa16d3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1681a0640 con 0x7fa168100930 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34892/0 (socket says 172.21.15.32:34892) 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 -- 172.21.15.32:0/851989612 learned_addr learned my addr 172.21.15.32:0/851989612 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa166ffd700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa168100930 0x7fa1681987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1677fe700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1681024c0 0x7fa16819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 -- 172.21.15.32:0/851989612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa168100930 msgr2=0x7fa1681987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa168100930 0x7fa1681987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 -- 172.21.15.32:0/851989612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1681024c0 msgr2=0x7fa16819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1681024c0 0x7fa16819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.426+0000 7fa1667fc700 1 -- 172.21.15.32:0/851989612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c0095f0 con 0x7fa168101b20 2023-08-11T15:44:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.427+0000 7fa1667fc700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168198d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009fd0 tx=0x7fa15c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.427+0000 7fa14ffff700 1 -- 172.21.15.32:0/851989612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c01d070 con 0x7fa168101b20 2023-08-11T15:44:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.427+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1681a08c0 con 0x7fa168101b20 2023-08-11T15:44:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.427+0000 7fa14ffff700 1 -- 172.21.15.32:0/851989612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15c005050 con 0x7fa168101b20 2023-08-11T15:44:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.427+0000 7fa14ffff700 1 -- 172.21.15.32:0/851989612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004b30 con 0x7fa168101b20 2023-08-11T15:44:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.427+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1681a0d40 con 0x7fa168101b20 2023-08-11T15:44:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.428+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168192f80 con 0x7fa168101b20 2023-08-11T15:44:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.430+0000 7fa14ffff700 1 -- 172.21.15.32:0/851989612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fa15c004c90 con 0x7fa168101b20 2023-08-11T15:44:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.431+0000 7fa14ffff700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa15009ffa0 0x7fa1500a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.432+0000 7fa14ffff700 1 -- 172.21.15.32:0/851989612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa15c0cc250 con 0x7fa168101b20 2023-08-11T15:44:26.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.432+0000 7fa166ffd700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa15009ffa0 0x7fa1500a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.432+0000 7fa166ffd700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa15009ffa0 0x7fa1500a2460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa168101980 tx=0x7fa158009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:26.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.435+0000 7fa14ffff700 1 -- 172.21.15.32:0/851989612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa15c095e20 con 0x7fa168101b20 2023-08-11T15:44:26.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:26 smithi134 ceph-mon[130447]: osdmap e88: 12 total, 12 up, 12 in 2023-08-11T15:44:26.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:26 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/313516958' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-11T15:44:26.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:26 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:26 smithi032 ceph-mon[97103]: osdmap e88: 12 total, 12 up, 12 in 2023-08-11T15:44:26.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:26 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/313516958' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-11T15:44:26.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:26 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:26 smithi192 ceph-mon[130806]: osdmap e88: 12 total, 12 up, 12 in 2023-08-11T15:44:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:26 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/313516958' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-11T15:44:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:26 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.644+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa16802cc10 con 0x7fa168101b20 2023-08-11T15:44:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.646+0000 7fa14ffff700 1 -- 172.21.15.32:0/851989612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23934 (secure 0 0 0) 0x7fa15c099440 con 0x7fa168101b20 2023-08-11T15:44:26.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:26.652 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:24.701219+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:26.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa15009ffa0 msgr2=0x7fa1500a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa15009ffa0 0x7fa1500a2460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa168101980 tx=0x7fa158009260 comp rx=0 tx=0).stop 2023-08-11T15:44:26.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 msgr2=0x7fa168198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168198d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009fd0 tx=0x7fa15c00f690 comp rx=0 tx=0).stop 2023-08-11T15:44:26.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 shutdown_connections 2023-08-11T15:44:26.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa15009ffa0 0x7fa1500a2460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1681024c0 0x7fa16819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa168100930 0x7fa1681987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 --2- 172.21.15.32:0/851989612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa168101b20 0x7fa168198d30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 >> 172.21.15.32:0/851989612 conn(0x7fa1680fbe60 msgr2=0x7fa1681073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:26.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 shutdown_connections 2023-08-11T15:44:26.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.649+0000 7fa16d3ff700 1 -- 172.21.15.32:0/851989612 wait complete. 2023-08-11T15:44:26.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 -- 172.21.15.32:0/948238071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30101b10 msgr2=0x7fed30101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 --2- 172.21.15.32:0/948238071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30101b10 0x7fed30101f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed200098f0 tx=0x7fed20009c00 comp rx=0 tx=0).stop 2023-08-11T15:44:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 -- 172.21.15.32:0/948238071 shutdown_connections 2023-08-11T15:44:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 --2- 172.21.15.32:0/948238071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 0x7fed30108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 --2- 172.21.15.32:0/948238071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed30100920 0x7fed30100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 --2- 172.21.15.32:0/948238071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30101b10 0x7fed30101f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 -- 172.21.15.32:0/948238071 >> 172.21.15.32:0/948238071 conn(0x7fed300fbe50 msgr2=0x7fed300fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 -- 172.21.15.32:0/948238071 shutdown_connections 2023-08-11T15:44:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.874+0000 7fed37fff700 1 -- 172.21.15.32:0/948238071 wait complete. 2023-08-11T15:44:26.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.875+0000 7fed37fff700 1 Processor -- start 2023-08-11T15:44:26.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.876+0000 7fed37fff700 1 -- start start 2023-08-11T15:44:26.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.876+0000 7fed37fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30100920 0x7fed30196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.876+0000 7fed37fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed30101b10 0x7fed30196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.876+0000 7fed37fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 0x7fed3019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.876+0000 7fed37fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3019e1a0 con 0x7fed301024b0 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.876+0000 7fed37fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3019e2e0 con 0x7fed30101b10 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.876+0000 7fed37fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3019e470 con 0x7fed30100920 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 0x7fed3019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 0x7fed3019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34920/0 (socket says 172.21.15.32:34920) 2023-08-11T15:44:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 -- 172.21.15.32:0/2223680247 learned_addr learned my addr 172.21.15.32:0/2223680247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3559a700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed30101b10 0x7fed30196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed35d9b700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30100920 0x7fed30196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 -- 172.21.15.32:0/2223680247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30100920 msgr2=0x7fed30196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30100920 0x7fed30196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 -- 172.21.15.32:0/2223680247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed30101b10 msgr2=0x7fed30196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed30101b10 0x7fed30196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 -- 172.21.15.32:0/2223680247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0095f0 con 0x7fed301024b0 2023-08-11T15:44:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed3659c700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 0x7fed3019dbd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed2800ef30 tx=0x7fed2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed26ffd700 1 -- 172.21.15.32:0/2223680247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28018070 con 0x7fed301024b0 2023-08-11T15:44:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed20009520 con 0x7fed301024b0 2023-08-11T15:44:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed26ffd700 1 -- 172.21.15.32:0/2223680247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed2800f040 con 0x7fed301024b0 2023-08-11T15:44:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.877+0000 7fed26ffd700 1 -- 172.21.15.32:0/2223680247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800cd00 con 0x7fed301024b0 2023-08-11T15:44:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.878+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3019ea70 con 0x7fed301024b0 2023-08-11T15:44:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.878+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3010c180 con 0x7fed301024b0 2023-08-11T15:44:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.880+0000 7fed26ffd700 1 -- 172.21.15.32:0/2223680247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fed28003a00 con 0x7fed301024b0 2023-08-11T15:44:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.882+0000 7fed26ffd700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fed1c09ffa0 0x7fed1c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.882+0000 7fed26ffd700 1 -- 172.21.15.32:0/2223680247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fed280cafe0 con 0x7fed301024b0 2023-08-11T15:44:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.882+0000 7fed35d9b700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fed1c09ffa0 0x7fed1c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.883+0000 7fed35d9b700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fed1c09ffa0 0x7fed1c0a2460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed30101970 tx=0x7fed2c009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:26.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:26.886+0000 7fed26ffd700 1 -- 172.21.15.32:0/2223680247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fed28094bb0 con 0x7fed301024b0 2023-08-11T15:44:27.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.099+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed3019ec90 con 0x7fed301024b0 2023-08-11T15:44:27.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.101+0000 7fed26ffd700 1 -- 172.21.15.32:0/2223680247 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23934 (secure 0 0 0) 0x7fed280981d0 con 0x7fed301024b0 2023-08-11T15:44:27.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:27.103 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:24.701219+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:27.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fed1c09ffa0 msgr2=0x7fed1c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fed1c09ffa0 0x7fed1c0a2460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed30101970 tx=0x7fed2c009310 comp rx=0 tx=0).stop 2023-08-11T15:44:27.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 msgr2=0x7fed3019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 0x7fed3019dbd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed2800ef30 tx=0x7fed2800c3a0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 shutdown_connections 2023-08-11T15:44:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fed1c09ffa0 0x7fed1c0a2460 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed301024b0 0x7fed3019dbd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed30100920 0x7fed30196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 --2- 172.21.15.32:0/2223680247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed30101b10 0x7fed30196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 >> 172.21.15.32:0/2223680247 conn(0x7fed300fbe50 msgr2=0x7fed300fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 shutdown_connections 2023-08-11T15:44:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.104+0000 7fed37fff700 1 -- 172.21.15.32:0/2223680247 wait complete. 2023-08-11T15:44:27.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/380012140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc8106640 msgr2=0x7f8dc8108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/380012140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc8106640 0x7f8dc8108a50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b0a0 tx=0x7f8dc000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/380012140 shutdown_connections 2023-08-11T15:44:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/380012140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc8106640 0x7f8dc8108a50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/380012140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc81013e0 0x7f8dc81037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/380012140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc8103d10 0x7f8dc8106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/380012140 >> 172.21.15.32:0/380012140 conn(0x7f8dc80fad50 msgr2=0x7f8dc80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/380012140 shutdown_connections 2023-08-11T15:44:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.349+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/380012140 wait complete. 2023-08-11T15:44:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.350+0000 7f8dcf7ff700 1 Processor -- start 2023-08-11T15:44:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcf7ff700 1 -- start start 2023-08-11T15:44:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcf7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc81013e0 0x7f8dc8100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcf7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc8103d10 0x7f8dc80ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcf7ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc8106640 0x7f8dc80ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcf7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8100070 con 0x7f8dc81013e0 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcf7ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc81001e0 con 0x7f8dc8106640 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcf7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc81a06d0 con 0x7f8dc8103d10 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc81013e0 0x7f8dc8100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc81013e0 0x7f8dc8100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34938/0 (socket says 172.21.15.32:34938) 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dccd9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc8103d10 0x7f8dc80ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 -- 172.21.15.32:0/2611142784 learned_addr learned my addr 172.21.15.32:0/2611142784 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcdd9c700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc8106640 0x7f8dc80ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 -- 172.21.15.32:0/2611142784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc8103d10 msgr2=0x7f8dc80ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc8103d10 0x7f8dc80ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 -- 172.21.15.32:0/2611142784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc8106640 msgr2=0x7f8dc80ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc8106640 0x7f8dc80ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcd59b700 1 -- 172.21.15.32:0/2611142784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0009d00 con 0x7f8dc81013e0 2023-08-11T15:44:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.351+0000 7f8dcdd9c700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc8106640 0x7f8dc80ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:44:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.352+0000 7f8dcd59b700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc81013e0 0x7f8dc8100d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e9a0 tx=0x7f8dc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.352+0000 7f8dbe7fc700 1 -- 172.21.15.32:0/2611142784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4009790 con 0x7f8dc81013e0 2023-08-11T15:44:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.352+0000 7f8dbe7fc700 1 -- 172.21.15.32:0/2611142784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dc400cbd0 con 0x7f8dc81013e0 2023-08-11T15:44:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.352+0000 7f8dbe7fc700 1 -- 172.21.15.32:0/2611142784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4018680 con 0x7f8dc81013e0 2023-08-11T15:44:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.352+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc81a09b0 con 0x7f8dc81013e0 2023-08-11T15:44:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.352+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc81a0e70 con 0x7f8dc81013e0 2023-08-11T15:44:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.353+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc80fc8d0 con 0x7f8dc81013e0 2023-08-11T15:44:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.355+0000 7f8dbe7fc700 1 -- 172.21.15.32:0/2611142784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f8dc400cd40 con 0x7f8dc81013e0 2023-08-11T15:44:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.357+0000 7f8dbe7fc700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8db40a0020 0x7f8db40a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.357+0000 7f8dbe7fc700 1 -- 172.21.15.32:0/2611142784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8dc4014070 con 0x7f8dc81013e0 2023-08-11T15:44:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.358+0000 7f8dccd9a700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8db40a0020 0x7f8db40a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.358+0000 7f8dccd9a700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8db40a0020 0x7f8db40a24e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009c60 tx=0x7f8db8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:27.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.361+0000 7f8dbe7fc700 1 -- 172.21.15.32:0/2611142784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8dc4051c30 con 0x7f8dc81013e0 2023-08-11T15:44:27.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:27 smithi134 ceph-mon[130447]: pgmap v274: 129 pgs: 2 creating+peering, 28 unknown, 99 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-08-11T15:44:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:27 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2098792582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:27 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/851989612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:27 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2223680247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:27 smithi032 ceph-mon[97103]: pgmap v274: 129 pgs: 2 creating+peering, 28 unknown, 99 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-08-11T15:44:27.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:27 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2098792582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:27 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/851989612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:27 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2223680247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:27 smithi192 ceph-mon[130806]: pgmap v274: 129 pgs: 2 creating+peering, 28 unknown, 99 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-08-11T15:44:27.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:27 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2098792582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:27 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/851989612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:27 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2223680247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.615+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dc802cba0 con 0x7f8dc81013e0 2023-08-11T15:44:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.616+0000 7f8dbe7fc700 1 -- 172.21.15.32:0/2611142784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11264 (secure 0 0 0) 0x7f8dc40951f0 con 0x7f8dc81013e0 2023-08-11T15:44:27.618 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:27.618 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:25.687892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-11T15:44:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.618+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8db40a0020 msgr2=0x7f8db40a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.618+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8db40a0020 0x7f8db40a24e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009c60 tx=0x7f8db8009260 comp rx=0 tx=0).stop 2023-08-11T15:44:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.618+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc81013e0 msgr2=0x7f8dc8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.618+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc81013e0 0x7f8dc8100d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e9a0 tx=0x7f8dc400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 shutdown_connections 2023-08-11T15:44:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f8db40a0020 0x7f8db40a24e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc81013e0 0x7f8dc8100d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc8106640 0x7f8dc80ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 --2- 172.21.15.32:0/2611142784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc8103d10 0x7f8dc80ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 >> 172.21.15.32:0/2611142784 conn(0x7f8dc80fad50 msgr2=0x7f8dc8102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 shutdown_connections 2023-08-11T15:44:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.619+0000 7f8dcf7ff700 1 -- 172.21.15.32:0/2611142784 wait complete. 2023-08-11T15:44:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2023-08-11T15:44:27.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 -- 172.21.15.32:0/390394564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724101b10 msgr2=0x7fb724101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 --2- 172.21.15.32:0/390394564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724101b10 0x7fb724101f70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb7100099c0 tx=0x7fb710009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 -- 172.21.15.32:0/390394564 shutdown_connections 2023-08-11T15:44:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 --2- 172.21.15.32:0/390394564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7241024b0 0x7fb724108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 --2- 172.21.15.32:0/390394564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb724100920 0x7fb724100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 --2- 172.21.15.32:0/390394564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724101b10 0x7fb724101f70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 -- 172.21.15.32:0/390394564 >> 172.21.15.32:0/390394564 conn(0x7fb7240fbe50 msgr2=0x7fb7240fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 -- 172.21.15.32:0/390394564 shutdown_connections 2023-08-11T15:44:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.842+0000 7fb72afff700 1 -- 172.21.15.32:0/390394564 wait complete. 2023-08-11T15:44:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 Processor -- start 2023-08-11T15:44:27.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 -- start start 2023-08-11T15:44:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724100920 0x7fb7240746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb724101b10 0x7fb724072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7241024b0 0x7fb724073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240739f0 con 0x7fb724100920 2023-08-11T15:44:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb724073b60 con 0x7fb724101b10 2023-08-11T15:44:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.843+0000 7fb72afff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7241a06c0 con 0x7fb7241024b0 2023-08-11T15:44:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724100920 0x7fb7240746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724100920 0x7fb7240746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34954/0 (socket says 172.21.15.32:34954) 2023-08-11T15:44:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 -- 172.21.15.32:0/3872340100 learned_addr learned my addr 172.21.15.32:0/3872340100 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb723fff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb724101b10 0x7fb724072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb72959c700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7241024b0 0x7fb724073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 -- 172.21.15.32:0/3872340100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7241024b0 msgr2=0x7fb724073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7241024b0 0x7fb724073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 -- 172.21.15.32:0/3872340100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb724101b10 msgr2=0x7fb724072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb724101b10 0x7fb724072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 -- 172.21.15.32:0/3872340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100095f0 con 0x7fb724100920 2023-08-11T15:44:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.844+0000 7fb728d9b700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724100920 0x7fb7240746d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb71800e970 tx=0x7fb71800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.845+0000 7fb721ffb700 1 -- 172.21.15.32:0/3872340100 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718009790 con 0x7fb724100920 2023-08-11T15:44:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.845+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7241a09a0 con 0x7fb724100920 2023-08-11T15:44:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.845+0000 7fb721ffb700 1 -- 172.21.15.32:0/3872340100 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb71800eec0 con 0x7fb724100920 2023-08-11T15:44:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.845+0000 7fb721ffb700 1 -- 172.21.15.32:0/3872340100 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718003cf0 con 0x7fb724100920 2023-08-11T15:44:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.845+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7241a0eb0 con 0x7fb724100920 2023-08-11T15:44:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.846+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb724107ec0 con 0x7fb724100920 2023-08-11T15:44:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.850+0000 7fb721ffb700 1 -- 172.21.15.32:0/3872340100 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fb71800ccd0 con 0x7fb724100920 2023-08-11T15:44:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.851+0000 7fb721ffb700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb71409ffa0 0x7fb7140a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.852+0000 7fb721ffb700 1 -- 172.21.15.32:0/3872340100 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb718014070 con 0x7fb724100920 2023-08-11T15:44:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.852+0000 7fb723fff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb71409ffa0 0x7fb7140a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.853+0000 7fb723fff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb71409ffa0 0x7fb7140a2460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb71000b580 tx=0x7fb710005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:27.854+0000 7fb721ffb700 1 -- 172.21.15.32:0/3872340100 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb7180953b0 con 0x7fb724100920 2023-08-11T15:44:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.073+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb72410c180 con 0x7fb724100920 2023-08-11T15:44:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.075+0000 7fb721ffb700 1 -- 172.21.15.32:0/3872340100 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23934 (secure 0 0 0) 0x7fb7180989d0 con 0x7fb724100920 2023-08-11T15:44:28.077 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:28.077 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:24.701219+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb71409ffa0 msgr2=0x7fb7140a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb71409ffa0 0x7fb7140a2460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb71000b580 tx=0x7fb710005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724100920 msgr2=0x7fb7240746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724100920 0x7fb7240746d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb71800e970 tx=0x7fb71800ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:28.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 shutdown_connections 2023-08-11T15:44:28.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb71409ffa0 0x7fb7140a2460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb724100920 0x7fb7240746d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7241024b0 0x7fb724073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 --2- 172.21.15.32:0/3872340100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb724101b10 0x7fb724072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.077+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 >> 172.21.15.32:0/3872340100 conn(0x7fb7240fbe50 msgr2=0x7fb724105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.078+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 shutdown_connections 2023-08-11T15:44:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.078+0000 7fb72afff700 1 -- 172.21.15.32:0/3872340100 wait complete. 2023-08-11T15:44:28.099 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-11T15:44:19.707016+0000', 'modified': '2023-08-11T15:44:25.687892+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/945169036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 945169036}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 945169036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-08-11T15:44:28.099 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-11T15:44:28.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T15:44:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3116644175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc4073380 msgr2=0x7fabc4073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3116644175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc4073380 0x7fabc4073800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009990 tx=0x7fabb4009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3116644175 shutdown_connections 2023-08-11T15:44:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3116644175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabc410cee0 0x7fabc410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3116644175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc4074960 0x7fabc4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3116644175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc4073380 0x7fabc4073800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3116644175 >> 172.21.15.32:0/3116644175 conn(0x7fabc40fbe60 msgr2=0x7fabc40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3116644175 shutdown_connections 2023-08-11T15:44:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.308+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3116644175 wait complete. 2023-08-11T15:44:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 Processor -- start 2023-08-11T15:44:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 -- start start 2023-08-11T15:44:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc4073380 0x7fabc419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabc4074960 0x7fabc419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc410cee0 0x7fabc41a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41a47e0 con 0x7fabc410cee0 2023-08-11T15:44:28.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41a4950 con 0x7fabc4074960 2023-08-11T15:44:28.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.309+0000 7fabcc5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41a4b10 con 0x7fabc4073380 2023-08-11T15:44:28.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc410cee0 0x7fabc41a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabca39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc4073380 0x7fabc419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc410cee0 0x7fabc41a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34980/0 (socket says 172.21.15.32:34980) 2023-08-11T15:44:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabcab9c700 1 -- 172.21.15.32:0/3712808679 learned_addr learned my addr 172.21.15.32:0/3712808679 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabc4074960 0x7fabc419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabca39b700 1 -- 172.21.15.32:0/3712808679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabc4074960 msgr2=0x7fabc419d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabca39b700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabc4074960 0x7fabc419d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabca39b700 1 -- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc410cee0 msgr2=0x7fabc41a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabca39b700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc410cee0 0x7fabc41a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabca39b700 1 -- 172.21.15.32:0/3712808679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40095f0 con 0x7fabc4073380 2023-08-11T15:44:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc410cee0 0x7fabc41a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.310+0000 7fabca39b700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc4073380 0x7fabc419cc70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e9f0 tx=0x7fabc000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.311+0000 7fabbb7fe700 1 -- 172.21.15.32:0/3712808679 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000c9c0 con 0x7fabc4073380 2023-08-11T15:44:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.311+0000 7fabbb7fe700 1 -- 172.21.15.32:0/3712808679 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabc0010430 con 0x7fabc4073380 2023-08-11T15:44:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.311+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc41a4df0 con 0x7fabc4073380 2023-08-11T15:44:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.311+0000 7fabbb7fe700 1 -- 172.21.15.32:0/3712808679 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0018680 con 0x7fabc4073380 2023-08-11T15:44:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.311+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41a5210 con 0x7fabc4073380 2023-08-11T15:44:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.314+0000 7fabbb7fe700 1 -- 172.21.15.32:0/3712808679 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fabc0010860 con 0x7fabc4073380 2023-08-11T15:44:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 Processor -- start 2023-08-11T15:44:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 -- start start 2023-08-11T15:44:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7af3e0 0x56542f7af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7afda0 0x56542f7b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f7075e0 con 0x56542f7af3e0 2023-08-11T15:44:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f704f00 con 0x56542f7afda0 2023-08-11T15:44:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.315+0000 7fabdff65b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f7007f0 con 0x56542f7aea80 2023-08-11T15:44:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabca39b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7af3e0 0x56542f7af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39600/0 (socket says 172.21.15.32:39600) 2023-08-11T15:44:28.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabcab9c700 1 -- 172.21.15.32:0/3887080348 learned_addr learned my addr 172.21.15.32:0/3887080348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:28.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabbb7fe700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fabb009ff50 0x7fabb00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7afda0 0x56542f7b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabbb7fe700 1 -- 172.21.15.32:0/3712808679 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fabc0014070 con 0x7fabc4073380 2023-08-11T15:44:28.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fabb009ff50 0x7fabb00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabca39b700 1 -- 172.21.15.32:0/3887080348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 msgr2=0x56542f7aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabca39b700 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabca39b700 1 -- 172.21.15.32:0/3887080348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7afda0 msgr2=0x56542f7b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabca39b700 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7afda0 0x56542f7b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabca39b700 1 -- 172.21.15.32:0/3887080348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56542f5805e0 con 0x56542f7af3e0 2023-08-11T15:44:28.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.316+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7afda0 0x56542f7b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:44:28.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.317+0000 7fabca39b700 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7af3e0 0x56542f7af860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fabc0098aa0 tx=0x7fabc0010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdcfdaf43b49f10 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.317+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fabb009ff50 0x7fabb00a2410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009120 tx=0x7fabbc004b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.317+0000 7fabb8bfe700 1 -- 172.21.15.32:0/3887080348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00ce970 con 0x56542f7af3e0 2023-08-11T15:44:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.317+0000 7fabb8bfe700 1 -- 172.21.15.32:0/3887080348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabc00988e0 con 0x56542f7af3e0 2023-08-11T15:44:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.317+0000 7fabb8bfe700 1 -- 172.21.15.32:0/3887080348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00a0c40 con 0x56542f7af3e0 2023-08-11T15:44:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.317+0000 7fabdff65b80 1 -- 172.21.15.32:0/3887080348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7af3e0 msgr2=0x56542f7af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.317+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7af3e0 0x56542f7af860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fabc0098aa0 tx=0x7fabc0010c60 comp rx=0 tx=0).stop 2023-08-11T15:44:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 -- 172.21.15.32:0/3887080348 shutdown_connections 2023-08-11T15:44:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7afda0 0x56542f7b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3887080348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7af3e0 0x56542f7af860 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 -- 172.21.15.32:0/3887080348 >> 172.21.15.32:0/3887080348 conn(0x56542f6db900 msgr2=0x56542f7a93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 -- 172.21.15.32:0/3887080348 shutdown_connections 2023-08-11T15:44:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 -- 172.21.15.32:0/3887080348 wait complete. 2023-08-11T15:44:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 -- start start 2023-08-11T15:44:28.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.318+0000 7fabdff65b80 1 Processor -- start 2023-08-11T15:44:28.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabdff65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f69ea70 con 0x56542f7afda0 2023-08-11T15:44:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabdff65b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f69e770 con 0x56542f7aea80 2023-08-11T15:44:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabdff65b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f5828d0 con 0x56542f7af3e0 2023-08-11T15:44:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabdff65b80 10 client.0 _ll_register_callbacks cb 0x56542f6a1a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:44:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34994/0 (socket says 172.21.15.32:34994) 2023-08-11T15:44:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabcab9c700 1 -- 172.21.15.32:0/3037491178 learned_addr learned my addr 172.21.15.32:0/3037491178 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.319+0000 7fabca39b700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabcab9c700 1 -- 172.21.15.32:0/3037491178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 msgr2=0x56542f7b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabcab9c700 1 -- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 msgr2=0x56542f7b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabcab9c700 1 -- 172.21.15.32:0/3037491178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41a5210 con 0x56542f7afda0 2023-08-11T15:44:28.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:28.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7bf610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fabbc019680 tx=0x7fabbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ad73c34e62fa9a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:28.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabadffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:44:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabadffb700 1 -- 172.21.15.32:0/3037491178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc017ee0 con 0x56542f7afda0 2023-08-11T15:44:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56542f7a1a40 con 0x56542f7afda0 2023-08-11T15:44:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabadffb700 1 -- 172.21.15.32:0/3037491178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fabbc019e20 con 0x56542f7afda0 2023-08-11T15:44:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabadffb700 1 -- 172.21.15.32:0/3037491178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabbc004bb0 con 0x56542f7afda0 2023-08-11T15:44:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabdff65b80 10 client.14667 fetch_fsmap learned FSMap version 17 2023-08-11T15:44:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.320+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56542f52cd90 con 0x56542f7afda0 2023-08-11T15:44:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabadffb700 1 -- 172.21.15.32:0/3037491178 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56542f7a1a40 con 0x56542f7afda0 2023-08-11T15:44:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabadffb700 1 -- 172.21.15.32:0/3037491178 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fabbc024400 con 0x56542f7afda0 2023-08-11T15:44:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabdff65b80 10 client.14667 fetch_fsmap finished waiting for FSMap version 17 2023-08-11T15:44:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabdff65b80 10 client.14667 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:44:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabdff65b80 20 client.14667 populate_metadata read hostname 'smithi032' 2023-08-11T15:44:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f7c3cb0 0x56542f7c2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabdff65b80 4 client.14667 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:44:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56542f77d490 con 0x56542f7c3cb0 2023-08-11T15:44:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.321+0000 7fabca39b700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f7c3cb0 0x56542f7c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.322+0000 7fabca39b700 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f7c3cb0 0x56542f7c2770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11fa03d7223c753d server_cookie=49ef0b6e31f3ea3b in_seq=0 out_seq=0 2023-08-11T15:44:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.322+0000 7fabadffb700 10 client.14667 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:44:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabadffb700 1 -- 172.21.15.32:0/3037491178 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x7fabc00b6670 con 0x56542f7c3cb0 2023-08-11T15:44:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabadffb700 10 client.14667 handle_command_reply: tid=0 2023-08-11T15:44:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 client.14667 shutdown 2023-08-11T15:44:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 msgr2=0x56542f7bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7bf610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fabbc019680 tx=0x7fabbc004ad0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 shutdown_connections 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f7c3cb0 0x56542f7c2770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7bf610 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabadffb700 0 client.14667 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3037491178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 >> 172.21.15.32:0/3037491178 conn(0x56542f6db900 msgr2=0x56542f7a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:28.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 shutdown_connections 2023-08-11T15:44:28.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.323+0000 7fabdff65b80 1 -- 172.21.15.32:0/3037491178 wait complete. 2023-08-11T15:44:28.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.324+0000 7fabdff65b80 20 client.14667 trim_cache size 0 max 16384 2023-08-11T15:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 Processor -- start 2023-08-11T15:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 -- start start 2023-08-11T15:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f69ad50 con 0x56542f7afda0 2023-08-11T15:44:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f6c9ef0 con 0x56542f7aea80 2023-08-11T15:44:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabdff65b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f6bd480 con 0x56542f7af3e0 2023-08-11T15:44:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35002/0 (socket says 172.21.15.32:35002) 2023-08-11T15:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabcab9c700 1 -- 172.21.15.32:0/3664586252 learned_addr learned my addr 172.21.15.32:0/3664586252 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.342+0000 7fabca39b700 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabcab9c700 1 -- 172.21.15.32:0/3664586252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 msgr2=0x56542f7b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabcab9c700 1 -- 172.21.15.32:0/3664586252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 msgr2=0x56542f7b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabcab9c700 1 -- 172.21.15.32:0/3664586252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56542f515c10 con 0x56542f7afda0 2023-08-11T15:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabcab9c700 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0247e0 tx=0x7fabbc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23eaf0f09b355f29 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabb93ff700 1 -- 172.21.15.32:0/3664586252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc017ec0 con 0x56542f7afda0 2023-08-11T15:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabb93ff700 1 -- 172.21.15.32:0/3664586252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabbc00f040 con 0x56542f7afda0 2023-08-11T15:44:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.343+0000 7fabb93ff700 1 -- 172.21.15.32:0/3664586252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc004e10 con 0x56542f7afda0 2023-08-11T15:44:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 -- 172.21.15.32:0/3664586252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 msgr2=0x56542f7cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0247e0 tx=0x7fabbc009510 comp rx=0 tx=0).stop 2023-08-11T15:44:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 -- 172.21.15.32:0/3664586252 shutdown_connections 2023-08-11T15:44:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc610 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7aea80 0x56542f7b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 --2- 172.21.15.32:0/3664586252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7af3e0 0x56542f7b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 -- 172.21.15.32:0/3664586252 >> 172.21.15.32:0/3664586252 conn(0x56542f6db900 msgr2=0x56542f7aa470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 -- 172.21.15.32:0/3664586252 shutdown_connections 2023-08-11T15:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 -- 172.21.15.32:0/3664586252 wait complete. 2023-08-11T15:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.344+0000 7fabdff65b80 1 -- start start 2023-08-11T15:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 1 Processor -- start 2023-08-11T15:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7af3e0 0x56542f7c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f7cc5a0 con 0x56542f7afda0 2023-08-11T15:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f7cc710 con 0x56542f7af3e0 2023-08-11T15:44:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542f7cc8d0 con 0x56542f7aea80 2023-08-11T15:44:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabdff65b80 10 client.0 _ll_register_callbacks cb 0x56542f6a1a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:44:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabcab9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35018/0 (socket says 172.21.15.32:35018) 2023-08-11T15:44:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabca39b700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabcab9c700 1 -- 172.21.15.32:0/2309564368 learned_addr learned my addr 172.21.15.32:0/2309564368 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.345+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7af3e0 0x56542f7c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabca39b700 1 -- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7af3e0 msgr2=0x56542f7c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabca39b700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7af3e0 0x56542f7c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabca39b700 1 -- 172.21.15.32:0/2309564368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 msgr2=0x56542f7cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:44:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabca39b700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabca39b700 1 -- 172.21.15.32:0/2309564368 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56542f52cd90 con 0x56542f7aea80 2023-08-11T15:44:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabcab9c700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabca39b700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7c6c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56542f7b07a0 tx=0x7fabc0010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e833c03f9b2da3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabadffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:44:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabadffb700 1 -- 172.21.15.32:0/2309564368 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0098e20 con 0x56542f7aea80 2023-08-11T15:44:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabadffb700 1 -- 172.21.15.32:0/2309564368 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fabc00994a0 con 0x56542f7aea80 2023-08-11T15:44:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabc00988e0 con 0x56542f7aea80 2023-08-11T15:44:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabadffb700 1 -- 172.21.15.32:0/2309564368 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabc0098700 con 0x56542f7aea80 2023-08-11T15:44:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabdff65b80 10 client.24685 fetch_fsmap learned FSMap version 17 2023-08-11T15:44:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.346+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56542f553200 con 0x56542f7aea80 2023-08-11T15:44:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabadffb700 1 -- 172.21.15.32:0/2309564368 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabc00988e0 con 0x56542f7aea80 2023-08-11T15:44:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabadffb700 1 -- 172.21.15.32:0/2309564368 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fabc00b5020 con 0x56542f7aea80 2023-08-11T15:44:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabdff65b80 10 client.24685 fetch_fsmap finished waiting for FSMap version 17 2023-08-11T15:44:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabdff65b80 10 client.24685 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:44:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabdff65b80 20 client.24685 populate_metadata read hostname 'smithi032' 2023-08-11T15:44:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabdff65b80 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f81c2f0 0x56542f818790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabdff65b80 4 client.24685 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-08-11T15:44:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56542f6f97d0 con 0x56542f81c2f0 2023-08-11T15:44:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.347+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f81c2f0 0x56542f818790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabc9b9a700 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f81c2f0 0x56542f818790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddbf75d912a9d152 server_cookie=ac3d52182abec0cd in_seq=0 out_seq=0 2023-08-11T15:44:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabadffb700 10 client.24685 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:44:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabadffb700 1 -- 172.21.15.32:0/2309564368 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56542f6f97d0 con 0x56542f81c2f0 2023-08-11T15:44:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabadffb700 10 client.24685 handle_command_reply: tid=0 2023-08-11T15:44:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 client.24685 shutdown 2023-08-11T15:44:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 msgr2=0x56542f7c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7c6c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56542f7b07a0 tx=0x7fabc0010c60 comp rx=0 tx=0).stop 2023-08-11T15:44:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 shutdown_connections 2023-08-11T15:44:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56542f81c2f0 0x56542f818790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56542f7afda0 0x56542f7cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56542f7aea80 0x56542f7c6c60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 --2- 172.21.15.32:0/2309564368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56542f7af3e0 0x56542f7c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabadffb700 0 client.24685 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:44:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 >> 172.21.15.32:0/2309564368 conn(0x56542f6db900 msgr2=0x56542f7b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 shutdown_connections 2023-08-11T15:44:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 1 -- 172.21.15.32:0/2309564368 wait complete. 2023-08-11T15:44:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.348+0000 7fabdff65b80 20 client.24685 trim_cache size 0 max 16384 2023-08-11T15:44:28.374 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:44:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "cd36a838-385c-11ee-9b37-001a4aab830c", 2023-08-11T15:44:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-08-11T15:44:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24578, 2023-08-11T15:44:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-08-11T15:44:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-08-11T15:44:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-08-11T15:44:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 8.1847464260000002, 2023-08-11T15:44:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 17, 2023-08-11T15:44:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 88, 2023-08-11T15:44:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 88, 2023-08-11T15:44:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 16.384122152 2023-08-11T15:44:28.377 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:44:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fabb009ff50 msgr2=0x7fabb00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fabb009ff50 0x7fabb00a2410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009120 tx=0x7fabbc004b40 comp rx=0 tx=0).stop 2023-08-11T15:44:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc4073380 msgr2=0x7fabc419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc4073380 0x7fabc419cc70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e9f0 tx=0x7fabc000ed00 comp rx=0 tx=0).stop 2023-08-11T15:44:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 shutdown_connections 2023-08-11T15:44:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fabb009ff50 0x7fabb00a2410 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc410cee0 0x7fabc41a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc4073380 0x7fabc419cc70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 --2- 172.21.15.32:0/3712808679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabc4074960 0x7fabc419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 >> 172.21.15.32:0/3712808679 conn(0x7fabc40fbe60 msgr2=0x7fabc40fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 shutdown_connections 2023-08-11T15:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:28.349+0000 7fabcc5ff700 1 -- 172.21.15.32:0/3712808679 wait complete. 2023-08-11T15:44:28.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:28 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2611142784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:28.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:28 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3872340100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:28.384 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T15:44:28.386 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:28.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph fs ls 2023-08-11T15:44:28.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:28 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2611142784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:28.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:28 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3872340100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:28.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:28 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2611142784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:28.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:28 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3872340100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:44:29.440 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:29 smithi032 ceph-mon[97103]: pgmap v275: 129 pgs: 2 creating+peering, 3 unknown, 124 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2023-08-11T15:44:29.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:29 smithi134 ceph-mon[130447]: pgmap v275: 129 pgs: 2 creating+peering, 3 unknown, 124 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2023-08-11T15:44:29.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:29 smithi192 ceph-mon[130806]: pgmap v275: 129 pgs: 2 creating+peering, 3 unknown, 124 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2023-08-11T15:44:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/1057884390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4101300 msgr2=0x7fd9b410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:30.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/1057884390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4101300 0x7fd9b410d810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b0a0 tx=0x7fd9ac00b3b0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/1057884390 shutdown_connections 2023-08-11T15:44:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/1057884390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4101300 0x7fd9b410d810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/1057884390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4106940 0x7fd9b4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/1057884390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b4100940 0x7fd9b4100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/1057884390 >> 172.21.15.32:0/1057884390 conn(0x7fd9b40fc410 msgr2=0x7fd9b40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:30.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/1057884390 shutdown_connections 2023-08-11T15:44:30.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.647+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/1057884390 wait complete. 2023-08-11T15:44:30.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.648+0000 7fd9bb5e2700 1 Processor -- start 2023-08-11T15:44:30.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.648+0000 7fd9bb5e2700 1 -- start start 2023-08-11T15:44:30.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9bb5e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4100940 0x7fd9b4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:30.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9bb5e2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101300 0x7fd9b41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:30.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9bb5e2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b4106940 0x7fd9b419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:30.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9bb5e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b419dd20 con 0x7fd9b4100940 2023-08-11T15:44:30.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9bb5e2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b419de90 con 0x7fd9b4101300 2023-08-11T15:44:30.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9bb5e2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b419e050 con 0x7fd9b4106940 2023-08-11T15:44:30.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4100940 0x7fd9b4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:30.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4100940 0x7fd9b4198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35046/0 (socket says 172.21.15.32:35046) 2023-08-11T15:44:30.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 -- 172.21.15.32:0/3068733907 learned_addr learned my addr 172.21.15.32:0/3068733907 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:30.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b8b7d700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101300 0x7fd9b41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:30.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b9b7f700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b4106940 0x7fd9b419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:30.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 -- 172.21.15.32:0/3068733907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b4106940 msgr2=0x7fd9b419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:30.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b4106940 0x7fd9b419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 -- 172.21.15.32:0/3068733907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101300 msgr2=0x7fd9b41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:30.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101300 0x7fd9b41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b937e700 1 -- 172.21.15.32:0/3068733907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac009d00 con 0x7fd9b4100940 2023-08-11T15:44:30.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.649+0000 7fd9b8b7d700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101300 0x7fd9b41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:44:30.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.650+0000 7fd9b937e700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4100940 0x7fd9b4198d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000e9a0 tx=0x7fd9b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:30.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.650+0000 7fd9aa7fc700 1 -- 172.21.15.32:0/3068733907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0009790 con 0x7fd9b4100940 2023-08-11T15:44:30.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.650+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b419e330 con 0x7fd9b4100940 2023-08-11T15:44:30.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.650+0000 7fd9aa7fc700 1 -- 172.21.15.32:0/3068733907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b000cbd0 con 0x7fd9b4100940 2023-08-11T15:44:30.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.650+0000 7fd9aa7fc700 1 -- 172.21.15.32:0/3068733907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0018680 con 0x7fd9b4100940 2023-08-11T15:44:30.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.650+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b41a5b90 con 0x7fd9b4100940 2023-08-11T15:44:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.651+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b404e620 con 0x7fd9b4100940 2023-08-11T15:44:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.653+0000 7fd9aa7fc700 1 -- 172.21.15.32:0/3068733907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fd9b000cd40 con 0x7fd9b4100940 2023-08-11T15:44:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.654+0000 7fd9aa7fc700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a009ff50 0x7fd9a00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.655+0000 7fd9aa7fc700 1 -- 172.21.15.32:0/3068733907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd9b0014070 con 0x7fd9b4100940 2023-08-11T15:44:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.655+0000 7fd9b8b7d700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a009ff50 0x7fd9a00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.655+0000 7fd9b8b7d700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a009ff50 0x7fd9a00a2410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9b419a380 tx=0x7fd9a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.658+0000 7fd9aa7fc700 1 -- 172.21.15.32:0/3068733907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd9b0051c30 con 0x7fd9b4100940 2023-08-11T15:44:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.911+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9b4110a80 con 0x7fd9b4100940 2023-08-11T15:44:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.911+0000 7fd9aa7fc700 1 -- 172.21.15.32:0/3068733907 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd9b0095210 con 0x7fd9b4100940 2023-08-11T15:44:30.913 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T15:44:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a009ff50 msgr2=0x7fd9a00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a009ff50 0x7fd9a00a2410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9b419a380 tx=0x7fd9a4009260 comp rx=0 tx=0).stop 2023-08-11T15:44:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4100940 msgr2=0x7fd9b4198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4100940 0x7fd9b4198d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000e9a0 tx=0x7fd9b000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 shutdown_connections 2023-08-11T15:44:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a009ff50 0x7fd9a00a2410 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b4100940 0x7fd9b4198d60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9b4106940 0x7fd9b419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 --2- 172.21.15.32:0/3068733907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101300 0x7fd9b41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 >> 172.21.15.32:0/3068733907 conn(0x7fd9b40fc410 msgr2=0x7fd9b410bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 shutdown_connections 2023-08-11T15:44:30.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:30.914+0000 7fd9bb5e2700 1 -- 172.21.15.32:0/3068733907 wait complete. 2023-08-11T15:44:31.397 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-11T15:44:31.398 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:44:31.398 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-08-11T15:44:31.398 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-08-11T15:44:31.399 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:31.399 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-11T15:44:31.399 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:44:31.399 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:44:31.399 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-08-11T15:44:31.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:31.400 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-08-11T15:44:31.446 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:44:31.446 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:44:31.446 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:44:31.446 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:44:31.447 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-08-11T15:44:31.447 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:44:31.447 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:44:31.447 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-08-11T15:44:31.447 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-08-11T15:44:31.447 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:44:31.447 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-08-11T15:44:31.448 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-08-11T15:44:31.448 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:44:31.448 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:44:31.448 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-08-11T15:44:31.448 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T15:44:31.448 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41662sec preferred_lft 41662sec 2023-08-11T15:44:31.448 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-08-11T15:44:31.449 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:44:31.449 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:44:31.450 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:44:31.450 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-08-11T15:44:31.451 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-11T15:44:31.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:31.452 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:44:31.452 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:44:31.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-08-11T15:44:31.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-08-11T15:44:31.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-08-11T15:44:31.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-11T15:44:31.453 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:44:31.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:31 smithi134 ceph-mon[130447]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s wr, 3 op/s 2023-08-11T15:44:31.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:31 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3068733907' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T15:44:31.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:31 smithi032 ceph-mon[97103]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s wr, 3 op/s 2023-08-11T15:44:31.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:31 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3068733907' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T15:44:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:31 smithi192 ceph-mon[130806]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 148 B/s wr, 3 op/s 2023-08-11T15:44:31.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:31 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3068733907' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T15:44:31.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:31.943 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:44:32.017 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-08-11T15:44:32.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:32.020 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-08-11T15:44:32.087 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-08-11T15:44:32.087 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:44:32.087 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T15:44:32.087 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T15:44:32.087 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:44:32.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:32.088 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:44:32.089 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:44:32.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T15:44:32.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T15:44:32.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T15:44:32.090 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:44:32.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:32.814 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-08-11T15:44:32.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:32.861 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-08-11T15:44:32.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:32.947 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:44:32.947 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:44:32.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-11T15:44:32.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-08-11T15:44:32.948 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:44:33.090 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:32 smithi032 ceph-mon[97103]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-08-11T15:44:33.133 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-08-11T15:44:33.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:33.134 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:44:33.135 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:44:33.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-08-11T15:44:33.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-11T15:44:33.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-08-11T15:44:33.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-08-11T15:44:33.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-08-11T15:44:33.136 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:44:33.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:32 smithi134 ceph-mon[130447]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-08-11T15:44:33.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:32 smithi192 ceph-mon[130806]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-08-11T15:44:33.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:33.452 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:44:33.452 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:44:33.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-08-11T15:44:33.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-08-11T15:44:33.454 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:44:33.633 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-08-11T15:44:33.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:44:33.634 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:33.679 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-08-11T15:44:33.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:44:33.680 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:33.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-08-11T15:44:33.875 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:44:33.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:44:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:44:33.948 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:44:33.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:33.949 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-08-11T15:44:33.993 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-11T15:44:33.994 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-08-11T15:44:34.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-08-11T15:44:34.113 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:44:34.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:44:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:44:34.188 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:44:34.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:34.190 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-08-11T15:44:34.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:2023-08-11T15:44:34.225+0000 7f916e4235c0 -1 init, newargv = 0x5605c95f9f20 newargc=15 2023-08-11T15:44:34.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[133662]: starting ceph client 2023-08-11T15:44:34.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[133662]: starting fuse 2023-08-11T15:44:35.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-08-11T15:44:35.406 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:35 smithi032 ceph-mon[97103]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 2 op/s 2023-08-11T15:44:35.417 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T15:44:35.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T15:44:35.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:35 smithi134 ceph-mon[130447]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 2 op/s 2023-08-11T15:44:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T15:44:35.491 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T15:44:35.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:35.493 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-08-11T15:44:35.536 INFO:teuthology.orchestra.run.smithi032.stdout:121 2023-08-11T15:44:35.536 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-08-11T15:44:35.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-08-11T15:44:35.537 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-08-11T15:44:35.538 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-08-11T15:44:35.538 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-08-11T15:44:35.538 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-08-11T15:44:35.538 DEBUG:teuthology.orchestra.run.smithi032:> 2023-08-11T15:44:35.540 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-08-11T15:44:35.541 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-08-11T15:44:35.541 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-08-11T15:44:35.542 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-08-11T15:44:35.542 DEBUG:teuthology.orchestra.run.smithi032:> 2023-08-11T15:44:35.542 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-08-11T15:44:35.543 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-08-11T15:44:35.543 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-08-11T15:44:35.543 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-08-11T15:44:35.543 DEBUG:teuthology.orchestra.run.smithi032:> 2023-08-11T15:44:35.544 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-08-11T15:44:35.545 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T15:44:35.546 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T15:44:35.546 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T15:44:35.546 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-08-11T15:44:35.547 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-08-11T15:44:35.547 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-08-11T15:44:35.547 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T15:44:35.548 DEBUG:teuthology.orchestra.run.smithi032:> 2023-08-11T15:44:35.548 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.admin")) 2023-08-11T15:44:35.548 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-08-11T15:44:35.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:35 smithi192 ceph-mon[130806]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 2 op/s 2023-08-11T15:44:35.721 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.admin.133662.asok 2023-08-11T15:44:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T15:44:35.735 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.133662.asok 2023-08-11T15:44:35.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:35.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.133662.asok status 2023-08-11T15:44:35.971 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:44:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-08-11T15:44:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T15:44:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T15:44:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "admin", 2023-08-11T15:44:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-08-11T15:44:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-08-11T15:44:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "133662", 2023-08-11T15:44:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-08-11T15:44:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:44:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-08-11T15:44:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-08-11T15:44:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 14685, 2023-08-11T15:44:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-08-11T15:44:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-08-11T15:44:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-08-11T15:44:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 14685 2023-08-11T15:44:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:44:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-08-11T15:44:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-08-11T15:44:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:44:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3601915708 2023-08-11T15:44:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:44:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:44:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-08-11T15:44:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-08-11T15:44:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-08-11T15:44:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3601915708 2023-08-11T15:44:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:44:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.14685 192.168.0.1:0/3601915708", 2023-08-11T15:44:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3601915708", 2023-08-11T15:44:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-08-11T15:44:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 17, 2023-08-11T15:44:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 88, 2023-08-11T15:44:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-08-11T15:44:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-08-11T15:44:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2023-08-11T15:44:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:44:35.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:35.985 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-08-11T15:44:35.986 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:36.032 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-08-11T15:44:36.032 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:36.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:36.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:36.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:36.145 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-08-11T15:44:36.380 INFO:teuthology.orchestra.run.smithi032.stdout:# file: . 2023-08-11T15:44:36.380 INFO:teuthology.orchestra.run.smithi032.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-08-11T15:44:36.381 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:36.381 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:44:36.382 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:44:36.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 -- 172.21.15.32:0/2287237331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 msgr2=0x7f05fc101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 --2- 172.21.15.32:0/2287237331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 0x7f05fc101f70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009920 tx=0x7f05e8009c30 comp rx=0 tx=0).stop 2023-08-11T15:44:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 -- 172.21.15.32:0/2287237331 shutdown_connections 2023-08-11T15:44:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 --2- 172.21.15.32:0/2287237331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc1024b0 0x7f05fc108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 --2- 172.21.15.32:0/2287237331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05fc100920 0x7f05fc100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 --2- 172.21.15.32:0/2287237331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 0x7f05fc101f70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 -- 172.21.15.32:0/2287237331 >> 172.21.15.32:0/2287237331 conn(0x7f05fc0fbe50 msgr2=0x7f05fc0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 -- 172.21.15.32:0/2287237331 shutdown_connections 2023-08-11T15:44:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.672+0000 7f06031ff700 1 -- 172.21.15.32:0/2287237331 wait complete. 2023-08-11T15:44:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.673+0000 7f06031ff700 1 Processor -- start 2023-08-11T15:44:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f06031ff700 1 -- start start 2023-08-11T15:44:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f06031ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc100920 0x7f05fc198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f06031ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 0x7f05fc198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f06031ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05fc1024b0 0x7f05fc19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f06031ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0400 con 0x7f05fc101b10 2023-08-11T15:44:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f06031ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0540 con 0x7f05fc100920 2023-08-11T15:44:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f06031ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a06d0 con 0x7f05fc1024b0 2023-08-11T15:44:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f0600f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc100920 0x7f05fc198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f05fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 0x7f05fc198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f0600f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc100920 0x7f05fc198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:60462/0 (socket says 172.21.15.32:60462) 2023-08-11T15:44:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f0600f9b700 1 -- 172.21.15.32:0/1169415591 learned_addr learned my addr 172.21.15.32:0/1169415591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.674+0000 7f060179c700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05fc1024b0 0x7f05fc19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f0600f9b700 1 -- 172.21.15.32:0/1169415591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05fc1024b0 msgr2=0x7f05fc19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f0600f9b700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05fc1024b0 0x7f05fc19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f0600f9b700 1 -- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 msgr2=0x7f05fc198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f0600f9b700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 0x7f05fc198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f0600f9b700 1 -- 172.21.15.32:0/1169415591 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e8009520 con 0x7f05fc100920 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f05fbfff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 0x7f05fc198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f0600f9b700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc100920 0x7f05fc198880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05f000e9f0 tx=0x7f05f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f05f9ffb700 1 -- 172.21.15.32:0/1169415591 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0004220 con 0x7f05fc100920 2023-08-11T15:44:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc1a09b0 con 0x7f05fc100920 2023-08-11T15:44:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f05f9ffb700 1 -- 172.21.15.32:0/1169415591 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05f0004380 con 0x7f05fc100920 2023-08-11T15:44:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f05f9ffb700 1 -- 172.21.15.32:0/1169415591 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0003be0 con 0x7f05fc100920 2023-08-11T15:44:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.675+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc1a0ec0 con 0x7f05fc100920 2023-08-11T15:44:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.676+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc192f70 con 0x7f05fc100920 2023-08-11T15:44:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.679+0000 7f05f9ffb700 1 -- 172.21.15.32:0/1169415591 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f05f0003d40 con 0x7f05fc100920 2023-08-11T15:44:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.680+0000 7f05f9ffb700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05ec09ffa0 0x7f05ec0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.680+0000 7f05f9ffb700 1 -- 172.21.15.32:0/1169415591 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f05f0014070 con 0x7f05fc100920 2023-08-11T15:44:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.680+0000 7f05fbfff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05ec09ffa0 0x7f05ec0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.681+0000 7f05fbfff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05ec09ffa0 0x7f05ec0a2460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05e800fa40 tx=0x7f05e8019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.682+0000 7f05f9ffb700 1 -- 172.21.15.32:0/1169415591 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f05f0095140 con 0x7f05fc100920 2023-08-11T15:44:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.899+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05fc10c180 con 0x7f05fc100920 2023-08-11T15:44:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.901+0000 7f05f9ffb700 1 -- 172.21.15.32:0/1169415591 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23934 (secure 0 0 0) 0x7f05f0098760 con 0x7f05fc100920 2023-08-11T15:44:36.902 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:36.903 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:24.701219+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:36.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05ec09ffa0 msgr2=0x7f05ec0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:36.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05ec09ffa0 0x7f05ec0a2460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05e800fa40 tx=0x7f05e8019040 comp rx=0 tx=0).stop 2023-08-11T15:44:36.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc100920 msgr2=0x7f05fc198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:36.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc100920 0x7f05fc198880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05f000e9f0 tx=0x7f05f000ed00 comp rx=0 tx=0).stop 2023-08-11T15:44:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 shutdown_connections 2023-08-11T15:44:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f05ec09ffa0 0x7f05ec0a2460 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05fc1024b0 0x7f05fc19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc100920 0x7f05fc198880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 --2- 172.21.15.32:0/1169415591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05fc101b10 0x7f05fc198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 >> 172.21.15.32:0/1169415591 conn(0x7f05fc0fbe50 msgr2=0x7f05fc107470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 shutdown_connections 2023-08-11T15:44:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:36.903+0000 7f06031ff700 1 -- 172.21.15.32:0/1169415591 wait complete. 2023-08-11T15:44:36.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:44:36.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:44:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 -- 172.21.15.32:0/1589969895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 msgr2=0x7f4320100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 --2- 172.21.15.32:0/1589969895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320100d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43100099c0 tx=0x7f4310009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 -- 172.21.15.32:0/1589969895 shutdown_connections 2023-08-11T15:44:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 --2- 172.21.15.32:0/1589969895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201024c0 0x7f4320108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 --2- 172.21.15.32:0/1589969895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320100d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 --2- 172.21.15.32:0/1589969895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4320101b20 0x7f4320101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 -- 172.21.15.32:0/1589969895 >> 172.21.15.32:0/1589969895 conn(0x7f43200fbea0 msgr2=0x7f43200fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:37.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 -- 172.21.15.32:0/1589969895 shutdown_connections 2023-08-11T15:44:37.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.133+0000 7f43267ff700 1 -- 172.21.15.32:0/1589969895 wait complete. 2023-08-11T15:44:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.134+0000 7f43267ff700 1 Processor -- start 2023-08-11T15:44:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.134+0000 7f43267ff700 1 -- start start 2023-08-11T15:44:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f43267ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f43267ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4320101b20 0x7f4320198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f43267ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201024c0 0x7f432019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f43267ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201a03f0 con 0x7f43201024c0 2023-08-11T15:44:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f43267ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201a0530 con 0x7f4320100930 2023-08-11T15:44:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f43267ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201a06c0 con 0x7f4320101b20 2023-08-11T15:44:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:60484/0 (socket says 172.21.15.32:60484) 2023-08-11T15:44:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 -- 172.21.15.32:0/148016273 learned_addr learned my addr 172.21.15.32:0/148016273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f4324d9c700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201024c0 0x7f432019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431f7fe700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4320101b20 0x7f4320198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 -- 172.21.15.32:0/148016273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4320101b20 msgr2=0x7f4320198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4320101b20 0x7f4320198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 -- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201024c0 msgr2=0x7f432019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201024c0 0x7f432019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.135+0000 7f431ffff700 1 -- 172.21.15.32:0/148016273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43100095f0 con 0x7f4320100930 2023-08-11T15:44:37.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.136+0000 7f431ffff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320198870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4310009fd0 tx=0x7f431000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:37.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.136+0000 7f431d7fa700 1 -- 172.21.15.32:0/148016273 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431001d070 con 0x7f4320100930 2023-08-11T15:44:37.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.136+0000 7f431d7fa700 1 -- 172.21.15.32:0/148016273 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4310005050 con 0x7f4320100930 2023-08-11T15:44:37.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.136+0000 7f431d7fa700 1 -- 172.21.15.32:0/148016273 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310004a70 con 0x7f4320100930 2023-08-11T15:44:37.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.136+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43201a0940 con 0x7f4320100930 2023-08-11T15:44:37.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.136+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43201a0d10 con 0x7f4320100930 2023-08-11T15:44:37.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.137+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432010c190 con 0x7f4320100930 2023-08-11T15:44:37.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.139+0000 7f431d7fa700 1 -- 172.21.15.32:0/148016273 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f4310005450 con 0x7f4320100930 2023-08-11T15:44:37.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.141+0000 7f431d7fa700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f430809ffd0 0x7f43080a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.141+0000 7f431d7fa700 1 -- 172.21.15.32:0/148016273 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f43100cc1d0 con 0x7f4320100930 2023-08-11T15:44:37.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.141+0000 7f431f7fe700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f430809ffd0 0x7f43080a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.141+0000 7f431f7fe700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f430809ffd0 0x7f43080a2490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4314009c60 tx=0x7f4314009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:37.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.142+0000 7f431d7fa700 1 -- 172.21.15.32:0/148016273 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4310095da0 con 0x7f4320100930 2023-08-11T15:44:37.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:37 smithi032 ceph-mon[97103]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-08-11T15:44:37.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:37 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1169415591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:37.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.357+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43201a0ff0 con 0x7f4320100930 2023-08-11T15:44:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.358+0000 7f431d7fa700 1 -- 172.21.15.32:0/148016273 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43100993c0 con 0x7f4320100930 2023-08-11T15:44:37.359 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:44:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f430809ffd0 msgr2=0x7f43080a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f430809ffd0 0x7f43080a2490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4314009c60 tx=0x7f4314009260 comp rx=0 tx=0).stop 2023-08-11T15:44:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 msgr2=0x7f4320198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320198870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4310009fd0 tx=0x7f431000f690 comp rx=0 tx=0).stop 2023-08-11T15:44:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 shutdown_connections 2023-08-11T15:44:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f430809ffd0 0x7f43080a2490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201024c0 0x7f432019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4320100930 0x7f4320198870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 --2- 172.21.15.32:0/148016273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4320101b20 0x7f4320198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 >> 172.21.15.32:0/148016273 conn(0x7f43200fbea0 msgr2=0x7f43200fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.360+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 shutdown_connections 2023-08-11T15:44:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.361+0000 7f43267ff700 1 -- 172.21.15.32:0/148016273 wait complete. 2023-08-11T15:44:37.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:44:37.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:37 smithi134 ceph-mon[130447]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-08-11T15:44:37.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:37 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1169415591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:37 smithi192 ceph-mon[130806]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-08-11T15:44:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:37 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1169415591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.586+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/2938285841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84101b20 msgr2=0x7f2a84101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.586+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/2938285841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84101b20 0x7f2a84101f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2a740099c0 tx=0x7f2a74009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.586+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/2938285841 shutdown_connections 2023-08-11T15:44:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.586+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/2938285841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a841024c0 0x7f2a84108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.586+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/2938285841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a84100930 0x7f2a84100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.586+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/2938285841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84101b20 0x7f2a84101f80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.586+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/2938285841 >> 172.21.15.32:0/2938285841 conn(0x7f2a840fbea0 msgr2=0x7f2a840fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.587+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/2938285841 shutdown_connections 2023-08-11T15:44:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.587+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/2938285841 wait complete. 2023-08-11T15:44:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 Processor -- start 2023-08-11T15:44:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 -- start start 2023-08-11T15:44:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84100930 0x7f2a84198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a84101b20 0x7f2a84198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a841024c0 0x7f2a8419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a0380 con 0x7f2a84100930 2023-08-11T15:44:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a04c0 con 0x7f2a84101b20 2023-08-11T15:44:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8b3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a0650 con 0x7f2a841024c0 2023-08-11T15:44:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.588+0000 7f2a8919b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84100930 0x7f2a84198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84100930 0x7f2a84198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60044/0 (socket says 172.21.15.32:60044) 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 -- 172.21.15.32:0/693837377 learned_addr learned my addr 172.21.15.32:0/693837377 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8899a700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a84101b20 0x7f2a84198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8999c700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a841024c0 0x7f2a8419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 -- 172.21.15.32:0/693837377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a841024c0 msgr2=0x7f2a8419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a841024c0 0x7f2a8419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 -- 172.21.15.32:0/693837377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a84101b20 msgr2=0x7f2a84198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a84101b20 0x7f2a84198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 -- 172.21.15.32:0/693837377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a740095f0 con 0x7f2a84100930 2023-08-11T15:44:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8899a700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a84101b20 0x7f2a84198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8919b700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84100930 0x7f2a84198800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000e970 tx=0x7f2a8000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a7a7fc700 1 -- 172.21.15.32:0/693837377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80009790 con 0x7f2a84100930 2023-08-11T15:44:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a7a7fc700 1 -- 172.21.15.32:0/693837377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a8000c7e0 con 0x7f2a84100930 2023-08-11T15:44:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.589+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a841a0930 con 0x7f2a84100930 2023-08-11T15:44:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.590+0000 7f2a7a7fc700 1 -- 172.21.15.32:0/693837377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80003c70 con 0x7f2a84100930 2023-08-11T15:44:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.590+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a841a0e90 con 0x7f2a84100930 2023-08-11T15:44:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.591+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8410c140 con 0x7f2a84100930 2023-08-11T15:44:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.593+0000 7f2a7a7fc700 1 -- 172.21.15.32:0/693837377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f2a8000cc50 con 0x7f2a84100930 2023-08-11T15:44:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.594+0000 7f2a7a7fc700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a7009ffd0 0x7f2a700a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.594+0000 7f2a7a7fc700 1 -- 172.21.15.32:0/693837377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2a80014070 con 0x7f2a84100930 2023-08-11T15:44:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.594+0000 7f2a8899a700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a7009ffd0 0x7f2a700a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.595+0000 7f2a8899a700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a7009ffd0 0x7f2a700a2490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b580 tx=0x7f2a74005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.596+0000 7f2a7a7fc700 1 -- 172.21.15.32:0/693837377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2a80095280 con 0x7f2a84100930 2023-08-11T15:44:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.805+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a841a0ac0 con 0x7f2a84100930 2023-08-11T15:44:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.805+0000 7f2a7a7fc700 1 -- 172.21.15.32:0/693837377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a841a0ac0 con 0x7f2a84100930 2023-08-11T15:44:37.807 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:44:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.807+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a7009ffd0 msgr2=0x7f2a700a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.807+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a7009ffd0 0x7f2a700a2490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b580 tx=0x7f2a74005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.807+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84100930 msgr2=0x7f2a84198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.807+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84100930 0x7f2a84198800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000e970 tx=0x7f2a8000ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.807+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 shutdown_connections 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.808+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a7009ffd0 0x7f2a700a2490 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.808+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a84100930 0x7f2a84198800 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.808+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a841024c0 0x7f2a8419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.808+0000 7f2a8b3ff700 1 --2- 172.21.15.32:0/693837377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a84101b20 0x7f2a84198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.808+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 >> 172.21.15.32:0/693837377 conn(0x7f2a840fbea0 msgr2=0x7f2a840fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.808+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 shutdown_connections 2023-08-11T15:44:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:37.808+0000 7f2a8b3ff700 1 -- 172.21.15.32:0/693837377 wait complete. 2023-08-11T15:44:37.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:44:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 -- 172.21.15.32:0/2354820767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 msgr2=0x7f26f8073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/2354820767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f8073800 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009990 tx=0x7f26e8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 -- 172.21.15.32:0/2354820767 shutdown_connections 2023-08-11T15:44:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/2354820767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f810cee0 0x7f26f810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/2354820767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f8074960 0x7f26f8072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/2354820767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f8073800 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 -- 172.21.15.32:0/2354820767 >> 172.21.15.32:0/2354820767 conn(0x7f26f80fbe60 msgr2=0x7f26f80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 -- 172.21.15.32:0/2354820767 shutdown_connections 2023-08-11T15:44:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.034+0000 7f26ff5ff700 1 -- 172.21.15.32:0/2354820767 wait complete. 2023-08-11T15:44:38.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.035+0000 7f26ff5ff700 1 Processor -- start 2023-08-11T15:44:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.035+0000 7f26ff5ff700 1 -- start start 2023-08-11T15:44:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26ff5ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26ff5ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f8074960 0x7f26f819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26ff5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f810cee0 0x7f26f81a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26ff5ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81a47c0 con 0x7f26f8073380 2023-08-11T15:44:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26ff5ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81a4930 con 0x7f26f8074960 2023-08-11T15:44:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26ff5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81a4af0 con 0x7f26f810cee0 2023-08-11T15:44:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60058/0 (socket says 172.21.15.32:60058) 2023-08-11T15:44:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 -- 172.21.15.32:0/3827064453 learned_addr learned my addr 172.21.15.32:0/3827064453 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fcb9a700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f8074960 0x7f26f819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fdb9c700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f810cee0 0x7f26f81a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 -- 172.21.15.32:0/3827064453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f810cee0 msgr2=0x7f26f81a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f810cee0 0x7f26f81a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 -- 172.21.15.32:0/3827064453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f8074960 msgr2=0x7f26f819d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f8074960 0x7f26f819d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.036+0000 7f26fd39b700 1 -- 172.21.15.32:0/3827064453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80095f0 con 0x7f26f8073380 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.037+0000 7f26fd39b700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f819cc80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f26f400ef10 tx=0x7f26f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.037+0000 7f26ee7fc700 1 -- 172.21.15.32:0/3827064453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400ca70 con 0x7f26f8073380 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.037+0000 7f26ee7fc700 1 -- 172.21.15.32:0/3827064453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f400f040 con 0x7f26f8073380 2023-08-11T15:44:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.037+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f81a4d70 con 0x7f26f8073380 2023-08-11T15:44:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.037+0000 7f26ee7fc700 1 -- 172.21.15.32:0/3827064453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4014480 con 0x7f26f8073380 2023-08-11T15:44:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.037+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f81a5210 con 0x7f26f8073380 2023-08-11T15:44:38.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.038+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f8102a30 con 0x7f26f8073380 2023-08-11T15:44:38.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.040+0000 7f26ee7fc700 1 -- 172.21.15.32:0/3827064453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f26f4007500 con 0x7f26f8073380 2023-08-11T15:44:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.041+0000 7f26ee7fc700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26e409ffd0 0x7f26e40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.042+0000 7f26ee7fc700 1 -- 172.21.15.32:0/3827064453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f26f40cb740 con 0x7f26f8073380 2023-08-11T15:44:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.042+0000 7f26fcb9a700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26e409ffd0 0x7f26e40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.042+0000 7f26fcb9a700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26e409ffd0 0x7f26e40a2490 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26e800b2e0 tx=0x7f26e8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.045+0000 7f26ee7fc700 1 -- 172.21.15.32:0/3827064453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f26f4095280 con 0x7f26f8073380 2023-08-11T15:44:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.257+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26f8112750 con 0x7f26f8073380 2023-08-11T15:44:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.257+0000 7f26ee7fc700 1 -- 172.21.15.32:0/3827064453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26f40988a0 con 0x7f26f8073380 2023-08-11T15:44:38.259 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:44:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.259+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26e409ffd0 msgr2=0x7f26e40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.259+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26e409ffd0 0x7f26e40a2490 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26e800b2e0 tx=0x7f26e8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.259+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 msgr2=0x7f26f819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.259+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f819cc80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f26f400ef10 tx=0x7f26f400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 shutdown_connections 2023-08-11T15:44:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26e409ffd0 0x7f26e40a2490 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f8073380 0x7f26f819cc80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f810cee0 0x7f26f81a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 --2- 172.21.15.32:0/3827064453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f8074960 0x7f26f819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 >> 172.21.15.32:0/3827064453 conn(0x7f26f80fbe60 msgr2=0x7f26f80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 shutdown_connections 2023-08-11T15:44:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.260+0000 7f26ff5ff700 1 -- 172.21.15.32:0/3827064453 wait complete. 2023-08-11T15:44:38.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:44:38.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:38 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/148016273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:38 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/693837377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:44:38.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 -- 172.21.15.32:0/3573963213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 msgr2=0x7f83f4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 --2- 172.21.15.32:0/3573963213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 0x7f83f4101f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f83e80099c0 tx=0x7f83e8009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 -- 172.21.15.32:0/3573963213 shutdown_connections 2023-08-11T15:44:38.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 --2- 172.21.15.32:0/3573963213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f41024c0 0x7f83f4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 --2- 172.21.15.32:0/3573963213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f4100930 0x7f83f4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 --2- 172.21.15.32:0/3573963213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 0x7f83f4101f80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 -- 172.21.15.32:0/3573963213 >> 172.21.15.32:0/3573963213 conn(0x7f83f40fbea0 msgr2=0x7f83f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:38.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 -- 172.21.15.32:0/3573963213 shutdown_connections 2023-08-11T15:44:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.490+0000 7f83f95ff700 1 -- 172.21.15.32:0/3573963213 wait complete. 2023-08-11T15:44:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.491+0000 7f83f95ff700 1 Processor -- start 2023-08-11T15:44:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.491+0000 7f83f95ff700 1 -- start start 2023-08-11T15:44:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f95ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f4100930 0x7f83f41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f95ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 0x7f83f4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f95ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f41024c0 0x7f83f419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f95ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41a0350 con 0x7f83f4101b20 2023-08-11T15:44:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f95ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41a0490 con 0x7f83f41024c0 2023-08-11T15:44:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f95ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41a0620 con 0x7f83f4100930 2023-08-11T15:44:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 0x7f83f4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f4100930 0x7f83f41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f41024c0 0x7f83f419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f4100930 0x7f83f41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:57898/0 (socket says 172.21.15.32:57898) 2023-08-11T15:44:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 0x7f83f4198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60070/0 (socket says 172.21.15.32:60070) 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f41024c0 0x7f83f419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:60536/0 (socket says 172.21.15.32:60536) 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.492+0000 7f83f2ffd700 1 -- 172.21.15.32:0/1632952196 learned_addr learned my addr 172.21.15.32:0/1632952196 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f2ffd700 1 -- 172.21.15.32:0/1632952196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f41024c0 msgr2=0x7f83f419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f2ffd700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f41024c0 0x7f83f419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f2ffd700 1 -- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 msgr2=0x7f83f4198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f2ffd700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 0x7f83f4198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f2ffd700 1 -- 172.21.15.32:0/1632952196 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80095f0 con 0x7f83f4100930 2023-08-11T15:44:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f37fe700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f41024c0 0x7f83f419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f2ffd700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f4100930 0x7f83f41987d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83e400e970 tx=0x7f83e400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83dbfff700 1 -- 172.21.15.32:0/1632952196 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4009790 con 0x7f83f4100930 2023-08-11T15:44:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83dbfff700 1 -- 172.21.15.32:0/1632952196 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83e400c7e0 con 0x7f83f4100930 2023-08-11T15:44:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f41a0900 con 0x7f83f4100930 2023-08-11T15:44:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83dbfff700 1 -- 172.21.15.32:0/1632952196 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4003c70 con 0x7f83f4100930 2023-08-11T15:44:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.493+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f41a0e60 con 0x7f83f4100930 2023-08-11T15:44:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.494+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f4192f30 con 0x7f83f4100930 2023-08-11T15:44:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.496+0000 7f83dbfff700 1 -- 172.21.15.32:0/1632952196 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f83e400cc50 con 0x7f83f4100930 2023-08-11T15:44:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.497+0000 7f83dbfff700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f83dc09ffd0 0x7f83dc0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.498+0000 7f83dbfff700 1 -- 172.21.15.32:0/1632952196 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f83e4014070 con 0x7f83f4100930 2023-08-11T15:44:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.498+0000 7f83f27fc700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f83dc09ffd0 0x7f83dc0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.498+0000 7f83f27fc700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f83dc09ffd0 0x7f83dc0a2490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f83e800b580 tx=0x7f83e8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.500+0000 7f83dbfff700 1 -- 172.21.15.32:0/1632952196 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f83e4095260 con 0x7f83f4100930 2023-08-11T15:44:38.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:38 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/148016273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:38.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:38 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/693837377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:44:38.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:38 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/148016273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:38.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:38 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/693837377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:44:38.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.713+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f83f41a0a90 con 0x7f83f4100930 2023-08-11T15:44:38.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.713+0000 7f83dbfff700 1 -- 172.21.15.32:0/1632952196 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f83f41a0a90 con 0x7f83f4100930 2023-08-11T15:44:38.715 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:44:38.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f83dc09ffd0 msgr2=0x7f83dc0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f83dc09ffd0 0x7f83dc0a2490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f83e800b580 tx=0x7f83e8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f4100930 msgr2=0x7f83f41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f4100930 0x7f83f41987d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83e400e970 tx=0x7f83e400ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 shutdown_connections 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f83dc09ffd0 0x7f83dc0a2490 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f41024c0 0x7f83f419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83f4100930 0x7f83f41987d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 --2- 172.21.15.32:0/1632952196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f4101b20 0x7f83f4198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 >> 172.21.15.32:0/1632952196 conn(0x7f83f40fbea0 msgr2=0x7f83f4107420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:38.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 shutdown_connections 2023-08-11T15:44:38.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.716+0000 7f83f95ff700 1 -- 172.21.15.32:0/1632952196 wait complete. 2023-08-11T15:44:38.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:44:38.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/1071501496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4101b00 msgr2=0x7fb4b4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/1071501496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4101b00 0x7fb4b4101f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00099c0 tx=0x7fb4a0009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/1071501496 shutdown_connections 2023-08-11T15:44:38.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/1071501496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4b41024a0 0x7fb4b4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/1071501496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b4100910 0x7fb4b4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/1071501496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b4101b00 0x7fb4b4101f60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/1071501496 >> 172.21.15.32:0/1071501496 conn(0x7fb4b40fbe60 msgr2=0x7fb4b40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/1071501496 shutdown_connections 2023-08-11T15:44:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.939+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/1071501496 wait complete. 2023-08-11T15:44:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 Processor -- start 2023-08-11T15:44:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 -- start start 2023-08-11T15:44:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4b4100910 0x7fb4b40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b4101b00 0x7fb4b4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b41024a0 0x7fb4b4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40739f0 con 0x7fb4b41024a0 2023-08-11T15:44:38.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4073b60 con 0x7fb4b4100910 2023-08-11T15:44:38.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.940+0000 7fb4bb1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b41a06b0 con 0x7fb4b4101b00 2023-08-11T15:44:38.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b979c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b41024a0 0x7fb4b4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b8f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4b4100910 0x7fb4b40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b8f9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4b4100910 0x7fb4b40746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:60556/0 (socket says 172.21.15.32:60556) 2023-08-11T15:44:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b8f9b700 1 -- 172.21.15.32:0/2247313465 learned_addr learned my addr 172.21.15.32:0/2247313465 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b3fff700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b4101b00 0x7fb4b4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b979c700 1 -- 172.21.15.32:0/2247313465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b4101b00 msgr2=0x7fb4b4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b979c700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b4101b00 0x7fb4b4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b979c700 1 -- 172.21.15.32:0/2247313465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4b4100910 msgr2=0x7fb4b40746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b979c700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4b4100910 0x7fb4b40746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b979c700 1 -- 172.21.15.32:0/2247313465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a00095f0 con 0x7fb4b41024a0 2023-08-11T15:44:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b979c700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b41024a0 0x7fb4b4073250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00efc0 tx=0x7fb4ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.941+0000 7fb4b1ffb700 1 -- 172.21.15.32:0/2247313465 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac018070 con 0x7fb4b41024a0 2023-08-11T15:44:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.942+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b41a0930 con 0x7fb4b41024a0 2023-08-11T15:44:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.942+0000 7fb4b1ffb700 1 -- 172.21.15.32:0/2247313465 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ac00f040 con 0x7fb4b41024a0 2023-08-11T15:44:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.942+0000 7fb4b1ffb700 1 -- 172.21.15.32:0/2247313465 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00cdf0 con 0x7fb4b41024a0 2023-08-11T15:44:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.942+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b41a0df0 con 0x7fb4b41024a0 2023-08-11T15:44:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.943+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b410c170 con 0x7fb4b41024a0 2023-08-11T15:44:38.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.945+0000 7fb4b1ffb700 1 -- 172.21.15.32:0/2247313465 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fb4ac007500 con 0x7fb4b41024a0 2023-08-11T15:44:38.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.946+0000 7fb4b1ffb700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4a409ff50 0x7fb4a40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:38.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.946+0000 7fb4b1ffb700 1 -- 172.21.15.32:0/2247313465 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb4ac0cb100 con 0x7fb4b41024a0 2023-08-11T15:44:38.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.946+0000 7fb4b8f9b700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4a409ff50 0x7fb4a40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:38.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.947+0000 7fb4b8f9b700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4a409ff50 0x7fb4a40a2410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8005950 tx=0x7fb4a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:38.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:38.950+0000 7fb4b1ffb700 1 -- 172.21.15.32:0/2247313465 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb4ac094cd0 con 0x7fb4b41024a0 2023-08-11T15:44:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.179+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4b410c350 con 0x7fb4b41024a0 2023-08-11T15:44:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.179+0000 7fb4b1ffb700 1 -- 172.21.15.32:0/2247313465 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+588 (secure 0 0 0) 0x7fb4ac0982f0 con 0x7fb4b41024a0 2023-08-11T15:44:39.182 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/33820350 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:39.182 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/4253467068 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:39.183 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3287657132 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:39.183 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3253045891 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:39.183 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3969802586 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:39.184 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/114266517 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:39.184 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/114266517 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:39.184 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/4253467068 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:39.185 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3000621007 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:39.185 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1572440619 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4a409ff50 msgr2=0x7fb4a40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4a409ff50 0x7fb4a40a2410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8005950 tx=0x7fb4a8009260 comp rx=0 tx=0).stop 2023-08-11T15:44:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b41024a0 msgr2=0x7fb4b4073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b41024a0 0x7fb4b4073250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00efc0 tx=0x7fb4ac00c3a0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 shutdown_connections 2023-08-11T15:44:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4a409ff50 0x7fb4a40a2410 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b41024a0 0x7fb4b4073250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4b4100910 0x7fb4b40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 --2- 172.21.15.32:0/2247313465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b4101b00 0x7fb4b4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 >> 172.21.15.32:0/2247313465 conn(0x7fb4b40fbe60 msgr2=0x7fb4b40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 shutdown_connections 2023-08-11T15:44:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.182+0000 7fb4bb1ff700 1 -- 172.21.15.32:0/2247313465 wait complete. 2023-08-11T15:44:39.189 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2023-08-11T15:44:39.202 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:44:39.203 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:44:39.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:39.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:39 smithi134 ceph-mon[130447]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:39.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:39 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3827064453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:44:39.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:39 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1632952196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:44:39.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 -- 172.21.15.32:0/3383122844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c1024b0 msgr2=0x7fc72c108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3383122844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c1024b0 0x7fc72c108d60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc72400b0d0 tx=0x7fc72400b3e0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 -- 172.21.15.32:0/3383122844 shutdown_connections 2023-08-11T15:44:39.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3383122844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c1024b0 0x7fc72c108d60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3383122844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c100920 0x7fc72c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3383122844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c101b10 0x7fc72c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 -- 172.21.15.32:0/3383122844 >> 172.21.15.32:0/3383122844 conn(0x7fc72c0fbe50 msgr2=0x7fc72c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:39.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 -- 172.21.15.32:0/3383122844 shutdown_connections 2023-08-11T15:44:39.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.492+0000 7fc7331ff700 1 -- 172.21.15.32:0/3383122844 wait complete. 2023-08-11T15:44:39.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.493+0000 7fc7331ff700 1 Processor -- start 2023-08-11T15:44:39.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc7331ff700 1 -- start start 2023-08-11T15:44:39.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc7331ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c100920 0x7fc72c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc7331ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c101b10 0x7fc72c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc7331ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c1024b0 0x7fc72c19dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc7331ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c19e260 con 0x7fc72c100920 2023-08-11T15:44:39.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc7331ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c19e3a0 con 0x7fc72c101b10 2023-08-11T15:44:39.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc7331ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c19e530 con 0x7fc72c1024b0 2023-08-11T15:44:39.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc730f9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c100920 0x7fc72c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc730f9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c100920 0x7fc72c1966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60102/0 (socket says 172.21.15.32:60102) 2023-08-11T15:44:39.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.494+0000 7fc730f9b700 1 -- 172.21.15.32:0/3466602013 learned_addr learned my addr 172.21.15.32:0/3466602013 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:39.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc72bfff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c101b10 0x7fc72c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc73179c700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c1024b0 0x7fc72c19dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc730f9b700 1 -- 172.21.15.32:0/3466602013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c1024b0 msgr2=0x7fc72c19dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc730f9b700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c1024b0 0x7fc72c19dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc730f9b700 1 -- 172.21.15.32:0/3466602013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c101b10 msgr2=0x7fc72c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc730f9b700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c101b10 0x7fc72c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc730f9b700 1 -- 172.21.15.32:0/3466602013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724009d00 con 0x7fc72c100920 2023-08-11T15:44:39.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc730f9b700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c100920 0x7fc72c1966e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc72000e970 tx=0x7fc72000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:39.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc729ffb700 1 -- 172.21.15.32:0/3466602013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720009790 con 0x7fc72c100920 2023-08-11T15:44:39.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72c19e810 con 0x7fc72c100920 2023-08-11T15:44:39.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc729ffb700 1 -- 172.21.15.32:0/3466602013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7200044e0 con 0x7fc72c100920 2023-08-11T15:44:39.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc729ffb700 1 -- 172.21.15.32:0/3466602013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720003e30 con 0x7fc72c100920 2023-08-11T15:44:39.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.495+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72c19ed20 con 0x7fc72c100920 2023-08-11T15:44:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.496+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72c10c190 con 0x7fc72c100920 2023-08-11T15:44:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.498+0000 7fc729ffb700 1 -- 172.21.15.32:0/3466602013 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fc720003f90 con 0x7fc72c100920 2023-08-11T15:44:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.500+0000 7fc729ffb700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc71c09ffa0 0x7fc71c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.500+0000 7fc729ffb700 1 -- 172.21.15.32:0/3466602013 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc720014070 con 0x7fc72c100920 2023-08-11T15:44:39.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.500+0000 7fc72bfff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc71c09ffa0 0x7fc71c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.500+0000 7fc72bfff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc71c09ffa0 0x7fc71c0a2460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc718009bf0 tx=0x7fc718009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:39.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.503+0000 7fc729ffb700 1 -- 172.21.15.32:0/3466602013 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc7200952d0 con 0x7fc72c100920 2023-08-11T15:44:39.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:39 smithi032 ceph-mon[97103]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:39.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:39 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3827064453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:44:39.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:39 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1632952196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:44:39.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:39 smithi192 ceph-mon[130806]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:39 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3827064453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:44:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:39 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1632952196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:44:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.716+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc72c10c3d0 con 0x7fc72c100920 2023-08-11T15:44:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.717+0000 7fc729ffb700 1 -- 172.21.15.32:0/3466602013 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23934 (secure 0 0 0) 0x7fc7200988f0 con 0x7fc72c100920 2023-08-11T15:44:39.720 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:39.720 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:24.701219+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.719+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc71c09ffa0 msgr2=0x7fc71c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.719+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc71c09ffa0 0x7fc71c0a2460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc718009bf0 tx=0x7fc718009260 comp rx=0 tx=0).stop 2023-08-11T15:44:39.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.719+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c100920 msgr2=0x7fc72c1966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.719+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c100920 0x7fc72c1966e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc72000e970 tx=0x7fc72000ed30 comp rx=0 tx=0).stop 2023-08-11T15:44:39.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 shutdown_connections 2023-08-11T15:44:39.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc71c09ffa0 0x7fc71c0a2460 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72c100920 0x7fc72c1966e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c1024b0 0x7fc72c19dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 --2- 172.21.15.32:0/3466602013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c101b10 0x7fc72c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 >> 172.21.15.32:0/3466602013 conn(0x7fc72c0fbe50 msgr2=0x7fc72c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 shutdown_connections 2023-08-11T15:44:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.720+0000 7fc7331ff700 1 -- 172.21.15.32:0/3466602013 wait complete. 2023-08-11T15:44:39.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:44:39.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:44:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 -- 172.21.15.32:0/1740173976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410101b20 msgr2=0x7f7410101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 --2- 172.21.15.32:0/1740173976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410101b20 0x7f7410101f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74000099c0 tx=0x7f7400009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 -- 172.21.15.32:0/1740173976 shutdown_connections 2023-08-11T15:44:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 --2- 172.21.15.32:0/1740173976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74101024c0 0x7f7410108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 --2- 172.21.15.32:0/1740173976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7410100930 0x7f7410100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 --2- 172.21.15.32:0/1740173976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410101b20 0x7f7410101f80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 -- 172.21.15.32:0/1740173976 >> 172.21.15.32:0/1740173976 conn(0x7f74100fbea0 msgr2=0x7f74100fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 -- 172.21.15.32:0/1740173976 shutdown_connections 2023-08-11T15:44:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.945+0000 7f7417fff700 1 -- 172.21.15.32:0/1740173976 wait complete. 2023-08-11T15:44:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 Processor -- start 2023-08-11T15:44:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 -- start start 2023-08-11T15:44:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410100930 0x7f7410198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7410101b20 0x7f7410198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74101024c0 0x7f741019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74101a03b0 con 0x7f7410100930 2023-08-11T15:44:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74101a04f0 con 0x7f74101024c0 2023-08-11T15:44:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.946+0000 7f7417fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74101a0680 con 0x7f7410101b20 2023-08-11T15:44:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410100930 0x7f7410198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410100930 0x7f7410198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60112/0 (socket says 172.21.15.32:60112) 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 -- 172.21.15.32:0/499112803 learned_addr learned my addr 172.21.15.32:0/499112803 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f741659c700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74101024c0 0x7f741019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f741559a700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7410101b20 0x7f7410198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 -- 172.21.15.32:0/499112803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7410101b20 msgr2=0x7f7410198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7410101b20 0x7f7410198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 -- 172.21.15.32:0/499112803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74101024c0 msgr2=0x7f741019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74101024c0 0x7f741019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.947+0000 7f7415d9b700 1 -- 172.21.15.32:0/499112803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74000095f0 con 0x7f7410100930 2023-08-11T15:44:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.948+0000 7f7415d9b700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410100930 0x7f7410198830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f740c00d980 tx=0x7f740c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.948+0000 7f7406ffd700 1 -- 172.21.15.32:0/499112803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00f890 con 0x7f7410100930 2023-08-11T15:44:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.948+0000 7f7406ffd700 1 -- 172.21.15.32:0/499112803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f740c010460 con 0x7f7410100930 2023-08-11T15:44:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.948+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74101a0960 con 0x7f7410100930 2023-08-11T15:44:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.948+0000 7f7406ffd700 1 -- 172.21.15.32:0/499112803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c0195c0 con 0x7f7410100930 2023-08-11T15:44:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.948+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74101a0e70 con 0x7f7410100930 2023-08-11T15:44:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.949+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741010c190 con 0x7f7410100930 2023-08-11T15:44:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.951+0000 7f7406ffd700 1 -- 172.21.15.32:0/499112803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f740c0105d0 con 0x7f7410100930 2023-08-11T15:44:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.952+0000 7f7406ffd700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f73fc09ff50 0x7f73fc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.952+0000 7f7406ffd700 1 -- 172.21.15.32:0/499112803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f740c0cb200 con 0x7f7410100930 2023-08-11T15:44:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.952+0000 7f741559a700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f73fc09ff50 0x7f73fc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.953+0000 7f741559a700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f73fc09ff50 0x7f73fc0a2410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7400006010 tx=0x7f7400005bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:39.956+0000 7f7406ffd700 1 -- 172.21.15.32:0/499112803 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f740c094d40 con 0x7f7410100930 2023-08-11T15:44:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.170+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74101a1060 con 0x7f7410100930 2023-08-11T15:44:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.170+0000 7f7406ffd700 1 -- 172.21.15.32:0/499112803 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f740c098360 con 0x7f7410100930 2023-08-11T15:44:40.172 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:44:40.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f73fc09ff50 msgr2=0x7f73fc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f73fc09ff50 0x7f73fc0a2410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7400006010 tx=0x7f7400005bc0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410100930 msgr2=0x7f7410198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410100930 0x7f7410198830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f740c00d980 tx=0x7f740c00dc90 comp rx=0 tx=0).stop 2023-08-11T15:44:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 shutdown_connections 2023-08-11T15:44:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f73fc09ff50 0x7f73fc0a2410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7410100930 0x7f7410198830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74101024c0 0x7f741019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 --2- 172.21.15.32:0/499112803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7410101b20 0x7f7410198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 >> 172.21.15.32:0/499112803 conn(0x7f74100fbea0 msgr2=0x7f74100fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 shutdown_connections 2023-08-11T15:44:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.173+0000 7f7417fff700 1 -- 172.21.15.32:0/499112803 wait complete. 2023-08-11T15:44:40.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:44:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.401+0000 7f86a3fff700 1 -- 172.21.15.32:0/2521802374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 msgr2=0x7f869c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.401+0000 7f86a3fff700 1 --2- 172.21.15.32:0/2521802374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c100d30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f869800b0d0 tx=0x7f869800b3e0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.401+0000 7f86a3fff700 1 -- 172.21.15.32:0/2521802374 shutdown_connections 2023-08-11T15:44:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.401+0000 7f86a3fff700 1 --2- 172.21.15.32:0/2521802374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c100d30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.401+0000 7f86a3fff700 1 --2- 172.21.15.32:0/2521802374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f869c1024a0 0x7f869c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.401+0000 7f86a3fff700 1 --2- 172.21.15.32:0/2521802374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869c101b00 0x7f869c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.402+0000 7f86a3fff700 1 -- 172.21.15.32:0/2521802374 >> 172.21.15.32:0/2521802374 conn(0x7f869c0fbe60 msgr2=0x7f869c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.402+0000 7f86a3fff700 1 -- 172.21.15.32:0/2521802374 shutdown_connections 2023-08-11T15:44:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.402+0000 7f86a3fff700 1 -- 172.21.15.32:0/2521802374 wait complete. 2023-08-11T15:44:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 Processor -- start 2023-08-11T15:44:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 -- start start 2023-08-11T15:44:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f869c101b00 0x7f869c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869c1024a0 0x7f869c19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c1a0550 con 0x7f869c100910 2023-08-11T15:44:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c1a0690 con 0x7f869c101b00 2023-08-11T15:44:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a3fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c1a0820 con 0x7f869c1024a0 2023-08-11T15:44:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60120/0 (socket says 172.21.15.32:60120) 2023-08-11T15:44:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.403+0000 7f86a1d9b700 1 -- 172.21.15.32:0/3736611187 learned_addr learned my addr 172.21.15.32:0/3736611187 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a159a700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f869c101b00 0x7f869c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a259c700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869c1024a0 0x7f869c19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a1d9b700 1 -- 172.21.15.32:0/3736611187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869c1024a0 msgr2=0x7f869c19ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a1d9b700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869c1024a0 0x7f869c19ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a1d9b700 1 -- 172.21.15.32:0/3736611187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f869c101b00 msgr2=0x7f869c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a1d9b700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f869c101b00 0x7f869c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a1d9b700 1 -- 172.21.15.32:0/3736611187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698009d00 con 0x7f869c100910 2023-08-11T15:44:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a1d9b700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c1989d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8698000f80 tx=0x7f8698008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f8692ffd700 1 -- 172.21.15.32:0/3736611187 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800e040 con 0x7f869c100910 2023-08-11T15:44:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f8692ffd700 1 -- 172.21.15.32:0/3736611187 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f869800be60 con 0x7f869c100910 2023-08-11T15:44:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c1a0aa0 con 0x7f869c100910 2023-08-11T15:44:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.404+0000 7f8692ffd700 1 -- 172.21.15.32:0/3736611187 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869801ba50 con 0x7f869c100910 2023-08-11T15:44:40.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.405+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c1a0f20 con 0x7f869c100910 2023-08-11T15:44:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.405+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c10c120 con 0x7f869c100910 2023-08-11T15:44:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.407+0000 7f8692ffd700 1 -- 172.21.15.32:0/3736611187 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f869801d670 con 0x7f869c100910 2023-08-11T15:44:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.410+0000 7f8692ffd700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f868809ff50 0x7f86880a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.410+0000 7f8692ffd700 1 -- 172.21.15.32:0/3736611187 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f86980cd200 con 0x7f869c100910 2023-08-11T15:44:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.410+0000 7f86a159a700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f868809ff50 0x7f86880a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.411+0000 7f86a159a700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f868809ff50 0x7f86880a2410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f868c009c60 tx=0x7f868c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.412+0000 7f8692ffd700 1 -- 172.21.15.32:0/3736611187 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8698096dd0 con 0x7f869c100910 2023-08-11T15:44:40.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:40 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2247313465' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:44:40.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:40 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3466602013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:40 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2247313465' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:44:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:40 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3466602013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:40.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:40 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2247313465' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:44:40.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:40 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3466602013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:40.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.625+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f869c1a10b0 con 0x7f869c100910 2023-08-11T15:44:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.625+0000 7f8692ffd700 1 -- 172.21.15.32:0/3736611187 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f869809a3f0 con 0x7f869c100910 2023-08-11T15:44:40.628 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:44:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f868809ff50 msgr2=0x7f86880a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f868809ff50 0x7f86880a2410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f868c009c60 tx=0x7f868c009260 comp rx=0 tx=0).stop 2023-08-11T15:44:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 msgr2=0x7f869c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c1989d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8698000f80 tx=0x7f8698008f10 comp rx=0 tx=0).stop 2023-08-11T15:44:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 shutdown_connections 2023-08-11T15:44:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f868809ff50 0x7f86880a2410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f869c100910 0x7f869c1989d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869c1024a0 0x7f869c19ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 --2- 172.21.15.32:0/3736611187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f869c101b00 0x7f869c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 >> 172.21.15.32:0/3736611187 conn(0x7f869c0fbe60 msgr2=0x7f869c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:40.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 shutdown_connections 2023-08-11T15:44:40.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.628+0000 7f86a3fff700 1 -- 172.21.15.32:0/3736611187 wait complete. 2023-08-11T15:44:40.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:44:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 -- 172.21.15.32:0/2858042004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8101b00 msgr2=0x7f81e8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 --2- 172.21.15.32:0/2858042004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8101b00 0x7f81e8101f60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0099c0 tx=0x7f81dc009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 -- 172.21.15.32:0/2858042004 shutdown_connections 2023-08-11T15:44:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 --2- 172.21.15.32:0/2858042004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81024a0 0x7f81e8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 --2- 172.21.15.32:0/2858042004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81e8100910 0x7f81e8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 --2- 172.21.15.32:0/2858042004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8101b00 0x7f81e8101f60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 -- 172.21.15.32:0/2858042004 >> 172.21.15.32:0/2858042004 conn(0x7f81e80fbe60 msgr2=0x7f81e80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 -- 172.21.15.32:0/2858042004 shutdown_connections 2023-08-11T15:44:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.855+0000 7f81edbff700 1 -- 172.21.15.32:0/2858042004 wait complete. 2023-08-11T15:44:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.856+0000 7f81edbff700 1 Processor -- start 2023-08-11T15:44:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.856+0000 7f81edbff700 1 -- start start 2023-08-11T15:44:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81edbff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8100910 0x7f81e80746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81edbff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81e8101b00 0x7f81e8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81edbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81024a0 0x7f81e8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81edbff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80739f0 con 0x7f81e8100910 2023-08-11T15:44:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81edbff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8073b60 con 0x7f81e8101b00 2023-08-11T15:44:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81edbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81a0690 con 0x7f81e81024a0 2023-08-11T15:44:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8100910 0x7f81e80746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8100910 0x7f81e80746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60142/0 (socket says 172.21.15.32:60142) 2023-08-11T15:44:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81e77fe700 1 -- 172.21.15.32:0/1649584847 learned_addr learned my addr 172.21.15.32:0/1649584847 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81e6ffd700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81e8101b00 0x7f81e8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.857+0000 7f81e7fff700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81024a0 0x7f81e8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e77fe700 1 -- 172.21.15.32:0/1649584847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81024a0 msgr2=0x7f81e8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e77fe700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81024a0 0x7f81e8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e77fe700 1 -- 172.21.15.32:0/1649584847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81e8101b00 msgr2=0x7f81e8072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e77fe700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81e8101b00 0x7f81e8072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e77fe700 1 -- 172.21.15.32:0/1649584847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc0095f0 con 0x7f81e8100910 2023-08-11T15:44:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e77fe700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8100910 0x7f81e80746d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f81d800e970 tx=0x7f81d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:40.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e4ff9700 1 -- 172.21.15.32:0/1649584847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8009790 con 0x7f81e8100910 2023-08-11T15:44:40.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e4ff9700 1 -- 172.21.15.32:0/1649584847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d800c7e0 con 0x7f81e8100910 2023-08-11T15:44:40.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81e4ff9700 1 -- 172.21.15.32:0/1649584847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8003c70 con 0x7f81e8100910 2023-08-11T15:44:40.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e81a0970 con 0x7f81e8100910 2023-08-11T15:44:40.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.858+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e81a0e30 con 0x7f81e8100910 2023-08-11T15:44:40.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.859+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e810c170 con 0x7f81e8100910 2023-08-11T15:44:40.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.861+0000 7f81e4ff9700 1 -- 172.21.15.32:0/1649584847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f81d800cc50 con 0x7f81e8100910 2023-08-11T15:44:40.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.863+0000 7f81e4ff9700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81d009ffd0 0x7f81d00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:40.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.863+0000 7f81e4ff9700 1 -- 172.21.15.32:0/1649584847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f81d8014070 con 0x7f81e8100910 2023-08-11T15:44:40.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.863+0000 7f81e6ffd700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81d009ffd0 0x7f81d00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:40.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.864+0000 7f81e6ffd700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81d009ffd0 0x7f81d00a2490 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009fd0 tx=0x7f81dc019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:40.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:40.867+0000 7f81e4ff9700 1 -- 172.21.15.32:0/1649584847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f81d8095110 con 0x7f81e8100910 2023-08-11T15:44:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.078+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81e8107840 con 0x7f81e8100910 2023-08-11T15:44:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.079+0000 7f81e4ff9700 1 -- 172.21.15.32:0/1649584847 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81d8003dd0 con 0x7f81e8100910 2023-08-11T15:44:41.080 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:44:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81d009ffd0 msgr2=0x7f81d00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81d009ffd0 0x7f81d00a2490 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009fd0 tx=0x7f81dc019040 comp rx=0 tx=0).stop 2023-08-11T15:44:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8100910 msgr2=0x7f81e80746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8100910 0x7f81e80746d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f81d800e970 tx=0x7f81d800ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 shutdown_connections 2023-08-11T15:44:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81d009ffd0 0x7f81d00a2490 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e8100910 0x7f81e80746d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81024a0 0x7f81e8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 --2- 172.21.15.32:0/1649584847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81e8101b00 0x7f81e8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.081+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 >> 172.21.15.32:0/1649584847 conn(0x7f81e80fbe60 msgr2=0x7f81e80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.082+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 shutdown_connections 2023-08-11T15:44:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.082+0000 7f81edbff700 1 -- 172.21.15.32:0/1649584847 wait complete. 2023-08-11T15:44:41.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:44:41.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 -- 172.21.15.32:0/3162683637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 msgr2=0x7f0120072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 --2- 172.21.15.32:0/3162683637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120072db0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0110009990 tx=0x7f0110009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 -- 172.21.15.32:0/3162683637 shutdown_connections 2023-08-11T15:44:41.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 --2- 172.21.15.32:0/3162683637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120072db0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 --2- 172.21.15.32:0/3162683637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01201087d0 0x7f012010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 --2- 172.21.15.32:0/3162683637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0120073380 0x7f0120073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 -- 172.21.15.32:0/3162683637 >> 172.21.15.32:0/3162683637 conn(0x7f01200fbe60 msgr2=0x7f01200fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 -- 172.21.15.32:0/3162683637 shutdown_connections 2023-08-11T15:44:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.308+0000 7f01269ff700 1 -- 172.21.15.32:0/3162683637 wait complete. 2023-08-11T15:44:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.309+0000 7f01269ff700 1 Processor -- start 2023-08-11T15:44:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f01269ff700 1 -- start start 2023-08-11T15:44:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f01269ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0120073380 0x7f0120198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f01269ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f01269ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01201087d0 0x7f012019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f01269ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201a0390 con 0x7f0120074960 2023-08-11T15:44:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f01269ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201a04d0 con 0x7f0120073380 2023-08-11T15:44:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f01269ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201a0660 con 0x7f01201087d0 2023-08-11T15:44:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f011f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f011f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60162/0 (socket says 172.21.15.32:60162) 2023-08-11T15:44:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f011f7fe700 1 -- 172.21.15.32:0/4204493823 learned_addr learned my addr 172.21.15.32:0/4204493823 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f011ffff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0120073380 0x7f0120198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.310+0000 7f0124f9c700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01201087d0 0x7f012019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011f7fe700 1 -- 172.21.15.32:0/4204493823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01201087d0 msgr2=0x7f012019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011f7fe700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01201087d0 0x7f012019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011f7fe700 1 -- 172.21.15.32:0/4204493823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0120073380 msgr2=0x7f0120198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011f7fe700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0120073380 0x7f0120198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011f7fe700 1 -- 172.21.15.32:0/4204493823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100095f0 con 0x7f0120074960 2023-08-11T15:44:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011ffff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0120073380 0x7f0120198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011f7fe700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120198d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f011400e970 tx=0x7f011400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011d7fa700 1 -- 172.21.15.32:0/4204493823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114009790 con 0x7f0120074960 2023-08-11T15:44:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011d7fa700 1 -- 172.21.15.32:0/4204493823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011400c7e0 con 0x7f0120074960 2023-08-11T15:44:41.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01201a0940 con 0x7f0120074960 2023-08-11T15:44:41.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f011d7fa700 1 -- 172.21.15.32:0/4204493823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114003c70 con 0x7f0120074960 2023-08-11T15:44:41.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.311+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01201a0e50 con 0x7f0120074960 2023-08-11T15:44:41.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.312+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0120106e80 con 0x7f0120074960 2023-08-11T15:44:41.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.314+0000 7f011d7fa700 1 -- 172.21.15.32:0/4204493823 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f011400cc50 con 0x7f0120074960 2023-08-11T15:44:41.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.316+0000 7f011d7fa700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f010809ff50 0x7f01080a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.316+0000 7f011d7fa700 1 -- 172.21.15.32:0/4204493823 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0114014070 con 0x7f0120074960 2023-08-11T15:44:41.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.316+0000 7f011ffff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f010809ff50 0x7f01080a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.317+0000 7f011ffff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f010809ff50 0x7f01080a2410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f011000b680 tx=0x7f0110005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:41.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.320+0000 7f011d7fa700 1 -- 172.21.15.32:0/4204493823 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0114095170 con 0x7f0120074960 2023-08-11T15:44:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:41 smithi134 ceph-mon[130447]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:41 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/499112803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:41 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3736611187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:41 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1649584847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.534+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f012002cd50 con 0x7f0120074960 2023-08-11T15:44:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.534+0000 7f011d7fa700 1 -- 172.21.15.32:0/4204493823 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0114003dd0 con 0x7f0120074960 2023-08-11T15:44:41.536 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:44:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.536+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f010809ff50 msgr2=0x7f01080a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.536+0000 7f01269ff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f010809ff50 0x7f01080a2410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f011000b680 tx=0x7f0110005fd0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.536+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 msgr2=0x7f0120198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.536+0000 7f01269ff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120198d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f011400e970 tx=0x7f011400ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 shutdown_connections 2023-08-11T15:44:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f010809ff50 0x7f01080a2410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01201087d0 0x7f012019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0120073380 0x7f0120198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 --2- 172.21.15.32:0/4204493823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0120074960 0x7f0120198d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 >> 172.21.15.32:0/4204493823 conn(0x7f01200fbe60 msgr2=0x7f01200fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 shutdown_connections 2023-08-11T15:44:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.537+0000 7f01269ff700 1 -- 172.21.15.32:0/4204493823 wait complete. 2023-08-11T15:44:41.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:41 smithi032 ceph-mon[97103]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:41.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:41 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/499112803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:41 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3736611187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:41 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1649584847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:44:41.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:41 smithi192 ceph-mon[130806]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:41.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:41 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/499112803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:41 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3736611187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:41 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1649584847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:44:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 -- 172.21.15.32:0/4239884549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 msgr2=0x7f2a20101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 --2- 172.21.15.32:0/4239884549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 0x7f2a20101f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a100098f0 tx=0x7f2a10009c00 comp rx=0 tx=0).stop 2023-08-11T15:44:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 -- 172.21.15.32:0/4239884549 shutdown_connections 2023-08-11T15:44:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 --2- 172.21.15.32:0/4239884549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 0x7f2a20108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 --2- 172.21.15.32:0/4239884549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a20100910 0x7f2a20100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 --2- 172.21.15.32:0/4239884549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 0x7f2a20101f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 -- 172.21.15.32:0/4239884549 >> 172.21.15.32:0/4239884549 conn(0x7f2a200fbe60 msgr2=0x7f2a200fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 -- 172.21.15.32:0/4239884549 shutdown_connections 2023-08-11T15:44:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.759+0000 7f2a281ff700 1 -- 172.21.15.32:0/4239884549 wait complete. 2023-08-11T15:44:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.760+0000 7f2a281ff700 1 Processor -- start 2023-08-11T15:44:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.760+0000 7f2a281ff700 1 -- start start 2023-08-11T15:44:41.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.760+0000 7f2a281ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a20100910 0x7f2a200746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.760+0000 7f2a281ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 0x7f2a20072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a281ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 0x7f2a20073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a281ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200739f0 con 0x7f2a201024a0 2023-08-11T15:44:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a281ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20073b60 con 0x7f2a20100910 2023-08-11T15:44:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a281ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a201a06c0 con 0x7f2a20101b00 2023-08-11T15:44:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 0x7f2a20073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 0x7f2a20073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60194/0 (socket says 172.21.15.32:60194) 2023-08-11T15:44:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 -- 172.21.15.32:0/3065856949 learned_addr learned my addr 172.21.15.32:0/3065856949 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a25f9b700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a20100910 0x7f2a200746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2579a700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 0x7f2a20072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 -- 172.21.15.32:0/3065856949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 msgr2=0x7f2a20072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 0x7f2a20072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 -- 172.21.15.32:0/3065856949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a20100910 msgr2=0x7f2a200746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a20100910 0x7f2a200746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.761+0000 7f2a2679c700 1 -- 172.21.15.32:0/3065856949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18009d00 con 0x7f2a201024a0 2023-08-11T15:44:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.762+0000 7f2a2679c700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 0x7f2a20073250 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800e3c0 tx=0x7f2a1800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.762+0000 7f2a16ffd700 1 -- 172.21.15.32:0/3065856949 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180090d0 con 0x7f2a201024a0 2023-08-11T15:44:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.762+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a10009520 con 0x7f2a201024a0 2023-08-11T15:44:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.762+0000 7f2a16ffd700 1 -- 172.21.15.32:0/3065856949 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a1800f040 con 0x7f2a201024a0 2023-08-11T15:44:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.762+0000 7f2a16ffd700 1 -- 172.21.15.32:0/3065856949 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18007c20 con 0x7f2a201024a0 2023-08-11T15:44:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.762+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a201a0d20 con 0x7f2a201024a0 2023-08-11T15:44:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.763+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2010c120 con 0x7f2a201024a0 2023-08-11T15:44:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.765+0000 7f2a16ffd700 1 -- 172.21.15.32:0/3065856949 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f2a18003a00 con 0x7f2a201024a0 2023-08-11T15:44:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.766+0000 7f2a16ffd700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a0c09ffd0 0x7f2a0c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.767+0000 7f2a16ffd700 1 -- 172.21.15.32:0/3065856949 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2a180cb1c0 con 0x7f2a201024a0 2023-08-11T15:44:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.767+0000 7f2a25f9b700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a0c09ffd0 0x7f2a0c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.767+0000 7f2a25f9b700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a0c09ffd0 0x7f2a0c0a2490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a20101960 tx=0x7f2a1c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.770+0000 7f2a16ffd700 1 -- 172.21.15.32:0/3065856949 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2a18094d90 con 0x7f2a201024a0 2023-08-11T15:44:41.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.997+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a2010c2e0 con 0x7f2a201024a0 2023-08-11T15:44:41.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:41.997+0000 7f2a16ffd700 1 -- 172.21.15.32:0/3065856949 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+588 (secure 0 0 0) 0x7f2a180983b0 con 0x7f2a201024a0 2023-08-11T15:44:42.000 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/33820350 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:42.000 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/4253467068 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:42.001 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3287657132 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:42.001 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3253045891 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:42.001 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3969802586 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:42.002 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/114266517 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:42.002 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/114266517 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:42.002 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/4253467068 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:42.003 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3000621007 2023-08-12T15:37:27.338674+0000 2023-08-11T15:44:42.003 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1572440619 2023-08-12T15:37:53.986517+0000 2023-08-11T15:44:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a0c09ffd0 msgr2=0x7f2a0c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a0c09ffd0 0x7f2a0c0a2490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a20101960 tx=0x7f2a1c009260 comp rx=0 tx=0).stop 2023-08-11T15:44:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 msgr2=0x7f2a20073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 0x7f2a20073250 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800e3c0 tx=0x7f2a1800e6d0 comp rx=0 tx=0).stop 2023-08-11T15:44:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 shutdown_connections 2023-08-11T15:44:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a0c09ffd0 0x7f2a0c0a2490 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a201024a0 0x7f2a20073250 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a20100910 0x7f2a200746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 --2- 172.21.15.32:0/3065856949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20101b00 0x7f2a20072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 >> 172.21.15.32:0/3065856949 conn(0x7f2a200fbe60 msgr2=0x7f2a200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 shutdown_connections 2023-08-11T15:44:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:42.000+0000 7f2a281ff700 1 -- 172.21.15.32:0/3065856949 wait complete. 2023-08-11T15:44:42.008 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2023-08-11T15:44:42.019 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-08-11T15:44:42.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:42.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-08-11T15:44:42.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[133662]: fuse finished with error 0 and tester_r 0 2023-08-11T15:44:42.095 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T15:44:42.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:42 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4204493823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:44:42.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:42 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3065856949' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:44:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:42 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4204493823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:44:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:42 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3065856949' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:44:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:42 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4204493823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:44:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:42 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3065856949' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:44:43.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:43 smithi134 ceph-mon[130447]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:43.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:43 smithi032 ceph-mon[97103]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:43.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:43 smithi192 ceph-mon[130806]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T15:44:45.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:45 smithi134 ceph-mon[130447]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:45 smithi032 ceph-mon[97103]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:45.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:45 smithi192 ceph-mon[130806]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:47 smithi134 ceph-mon[130447]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:47.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:47 smithi032 ceph-mon[97103]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:47.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:47 smithi192 ceph-mon[130806]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:48.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:44:48.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:44:48.198 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-08-11T15:44:48.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 -- 172.21.15.32:0/389468701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 msgr2=0x7fe4ac101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/389468701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac101f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00099c0 tx=0x7fe4a0009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 -- 172.21.15.32:0/389468701 shutdown_connections 2023-08-11T15:44:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/389468701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac1024b0 0x7fe4ac108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/389468701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4ac100920 0x7fe4ac100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/389468701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac101f70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 -- 172.21.15.32:0/389468701 >> 172.21.15.32:0/389468701 conn(0x7fe4ac0fbe50 msgr2=0x7fe4ac0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 -- 172.21.15.32:0/389468701 shutdown_connections 2023-08-11T15:44:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.475+0000 7fe4b0fff700 1 -- 172.21.15.32:0/389468701 wait complete. 2023-08-11T15:44:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.476+0000 7fe4b0fff700 1 Processor -- start 2023-08-11T15:44:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4b0fff700 1 -- start start 2023-08-11T15:44:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4b0fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4ac100920 0x7fe4ac1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4b0fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4b0fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac1024b0 0x7fe4ac19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4b0fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac19e140 con 0x7fe4ac101b10 2023-08-11T15:44:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4b0fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac19e280 con 0x7fe4ac1024b0 2023-08-11T15:44:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4b0fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac19e410 con 0x7fe4ac100920 2023-08-11T15:44:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4a9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4a9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40122/0 (socket says 172.21.15.32:40122) 2023-08-11T15:44:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4a9d9b700 1 -- 172.21.15.32:0/767744541 learned_addr learned my addr 172.21.15.32:0/767744541 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.477+0000 7fe4aa59c700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4ac100920 0x7fe4ac1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4aad9d700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac1024b0 0x7fe4ac19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4a9d9b700 1 -- 172.21.15.32:0/767744541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4ac100920 msgr2=0x7fe4ac1965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4a9d9b700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4ac100920 0x7fe4ac1965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4a9d9b700 1 -- 172.21.15.32:0/767744541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac1024b0 msgr2=0x7fe4ac19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4a9d9b700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac1024b0 0x7fe4ac19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4a9d9b700 1 -- 172.21.15.32:0/767744541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a00095f0 con 0x7fe4ac101b10 2023-08-11T15:44:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4a9d9b700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac196b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000c010 tx=0x7fe4a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe49b7fe700 1 -- 172.21.15.32:0/767744541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a001d070 con 0x7fe4ac101b10 2023-08-11T15:44:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe49b7fe700 1 -- 172.21.15.32:0/767744541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4a000fb80 con 0x7fe4ac101b10 2023-08-11T15:44:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe49b7fe700 1 -- 172.21.15.32:0/767744541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00173f0 con 0x7fe4ac101b10 2023-08-11T15:44:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac19e690 con 0x7fe4ac101b10 2023-08-11T15:44:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.478+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac19eab0 con 0x7fe4ac101b10 2023-08-11T15:44:48.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.479+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac10c130 con 0x7fe4ac101b10 2023-08-11T15:44:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.481+0000 7fe49b7fe700 1 -- 172.21.15.32:0/767744541 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fe4a0022470 con 0x7fe4ac101b10 2023-08-11T15:44:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.483+0000 7fe49b7fe700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe4940a0020 0x7fe4940a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.483+0000 7fe49b7fe700 1 -- 172.21.15.32:0/767744541 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe4a00cc6d0 con 0x7fe4ac101b10 2023-08-11T15:44:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.483+0000 7fe4aa59c700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe4940a0020 0x7fe4940a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.484+0000 7fe4aa59c700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe4940a0020 0x7fe4940a24e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac101970 tx=0x7fe49c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.487+0000 7fe49b7fe700 1 -- 172.21.15.32:0/767744541 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe4a005cc30 con 0x7fe4ac101b10 2023-08-11T15:44:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.701+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4ac19ed90 con 0x7fe4ac101b10 2023-08-11T15:44:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.703+0000 7fe49b7fe700 1 -- 172.21.15.32:0/767744541 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23934 (secure 0 0 0) 0x7fe4a00962a0 con 0x7fe4ac101b10 2023-08-11T15:44:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:48.706 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:24.701219+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:48.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe4940a0020 msgr2=0x7fe4940a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe4940a0020 0x7fe4940a24e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac101970 tx=0x7fe49c009260 comp rx=0 tx=0).stop 2023-08-11T15:44:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 msgr2=0x7fe4ac196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac196b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000c010 tx=0x7fe4a000f690 comp rx=0 tx=0).stop 2023-08-11T15:44:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 shutdown_connections 2023-08-11T15:44:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe4940a0020 0x7fe4940a24e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac1024b0 0x7fe4ac19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4ac100920 0x7fe4ac1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 --2- 172.21.15.32:0/767744541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac101b10 0x7fe4ac196b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 >> 172.21.15.32:0/767744541 conn(0x7fe4ac0fbe50 msgr2=0x7fe4ac0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 shutdown_connections 2023-08-11T15:44:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.705+0000 7fe4b0fff700 1 -- 172.21.15.32:0/767744541 wait complete. 2023-08-11T15:44:48.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 -- 172.21.15.32:0/1813564239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 msgr2=0x7efeec101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 --2- 172.21.15.32:0/1813564239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 0x7efeec101f70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efee00099c0 tx=0x7efee0009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 -- 172.21.15.32:0/1813564239 shutdown_connections 2023-08-11T15:44:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 --2- 172.21.15.32:0/1813564239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec100920 0x7efeec100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 --2- 172.21.15.32:0/1813564239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec1024b0 0x7efeec108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 --2- 172.21.15.32:0/1813564239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 0x7efeec101f70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 -- 172.21.15.32:0/1813564239 >> 172.21.15.32:0/1813564239 conn(0x7efeec0fbe50 msgr2=0x7efeec0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 -- 172.21.15.32:0/1813564239 shutdown_connections 2023-08-11T15:44:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.930+0000 7efef13ff700 1 -- 172.21.15.32:0/1813564239 wait complete. 2023-08-11T15:44:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.931+0000 7efef13ff700 1 Processor -- start 2023-08-11T15:44:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efef13ff700 1 -- start start 2023-08-11T15:44:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efef13ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec100920 0x7efeec196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efef13ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 0x7efeec196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efef13ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec1024b0 0x7efeec19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efef13ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec19e180 con 0x7efeec1024b0 2023-08-11T15:44:48.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efef13ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec19e2c0 con 0x7efeec101b10 2023-08-11T15:44:48.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efef13ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec19e450 con 0x7efeec100920 2023-08-11T15:44:48.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efeeb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec1024b0 0x7efeec19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.932+0000 7efeeaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec100920 0x7efeec196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec100920 0x7efeec196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:54934/0 (socket says 172.21.15.32:54934) 2023-08-11T15:44:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec1024b0 0x7efeec19dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40146/0 (socket says 172.21.15.32:40146) 2023-08-11T15:44:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeaffd700 1 -- 172.21.15.32:0/1330955825 learned_addr learned my addr 172.21.15.32:0/1330955825 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeea7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 0x7efeec196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeb7fe700 1 -- 172.21.15.32:0/1330955825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec100920 msgr2=0x7efeec196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeb7fe700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec100920 0x7efeec196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeb7fe700 1 -- 172.21.15.32:0/1330955825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 msgr2=0x7efeec196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeb7fe700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 0x7efeec196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeb7fe700 1 -- 172.21.15.32:0/1330955825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc009520 con 0x7efeec1024b0 2023-08-11T15:44:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeaffd700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec100920 0x7efeec196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efeeb7fe700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec1024b0 0x7efeec19dbb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efee400efc0 tx=0x7efee400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efed3fff700 1 -- 172.21.15.32:0/1330955825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4018070 con 0x7efeec1024b0 2023-08-11T15:44:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.933+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee00095f0 con 0x7efeec1024b0 2023-08-11T15:44:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.934+0000 7efed3fff700 1 -- 172.21.15.32:0/1330955825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee400f040 con 0x7efeec1024b0 2023-08-11T15:44:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.934+0000 7efed3fff700 1 -- 172.21.15.32:0/1330955825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400cd40 con 0x7efeec1024b0 2023-08-11T15:44:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.934+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec072ed0 con 0x7efeec1024b0 2023-08-11T15:44:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.935+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec10c1b0 con 0x7efeec1024b0 2023-08-11T15:44:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.936+0000 7efed3fff700 1 -- 172.21.15.32:0/1330955825 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7efee4003a00 con 0x7efeec1024b0 2023-08-11T15:44:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.938+0000 7efed3fff700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7efed409ffa0 0x7efed40a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.938+0000 7efed3fff700 1 -- 172.21.15.32:0/1330955825 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efee40caff0 con 0x7efeec1024b0 2023-08-11T15:44:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.938+0000 7efeeaffd700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7efed409ffa0 0x7efed40a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.939+0000 7efeeaffd700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7efed409ffa0 0x7efed40a2460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efedc009fd0 tx=0x7efedc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:48.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:48.942+0000 7efed3fff700 1 -- 172.21.15.32:0/1330955825 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7efee40fdc30 con 0x7efeec1024b0 2023-08-11T15:44:49.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.157+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeec10c3d0 con 0x7efeec1024b0 2023-08-11T15:44:49.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.159+0000 7efed3fff700 1 -- 172.21.15.32:0/1330955825 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23934 (secure 0 0 0) 0x7efee4094be0 con 0x7efeec1024b0 2023-08-11T15:44:49.161 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:49.162 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:24.701219+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.161+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7efed409ffa0 msgr2=0x7efed40a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.161+0000 7efef13ff700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7efed409ffa0 0x7efed40a2460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efedc009fd0 tx=0x7efedc009260 comp rx=0 tx=0).stop 2023-08-11T15:44:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.161+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec1024b0 msgr2=0x7efeec19dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.161+0000 7efef13ff700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec1024b0 0x7efeec19dbb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efee400efc0 tx=0x7efee400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 shutdown_connections 2023-08-11T15:44:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7efed409ffa0 0x7efed40a2460 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec1024b0 0x7efeec19dbb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeec100920 0x7efeec196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 --2- 172.21.15.32:0/1330955825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeec101b10 0x7efeec196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 >> 172.21.15.32:0/1330955825 conn(0x7efeec0fbe50 msgr2=0x7efeec0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:49.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 shutdown_connections 2023-08-11T15:44:49.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.162+0000 7efef13ff700 1 -- 172.21.15.32:0/1330955825 wait complete. 2023-08-11T15:44:49.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-11T15:44:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 -- 172.21.15.32:0/2584648966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518100980 msgr2=0x7f4518100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 --2- 172.21.15.32:0/2584648966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518100980 0x7f4518100da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45080099c0 tx=0x7f4508009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 -- 172.21.15.32:0/2584648966 shutdown_connections 2023-08-11T15:44:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 --2- 172.21.15.32:0/2584648966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518102510 0x7f4518108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 --2- 172.21.15.32:0/2584648966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518100980 0x7f4518100da0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 --2- 172.21.15.32:0/2584648966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518101b70 0x7f4518101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 -- 172.21.15.32:0/2584648966 >> 172.21.15.32:0/2584648966 conn(0x7f45180fbef0 msgr2=0x7f45180fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 -- 172.21.15.32:0/2584648966 shutdown_connections 2023-08-11T15:44:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.388+0000 7f451c9ff700 1 -- 172.21.15.32:0/2584648966 wait complete. 2023-08-11T15:44:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.389+0000 7f451c9ff700 1 Processor -- start 2023-08-11T15:44:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451c9ff700 1 -- start start 2023-08-11T15:44:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451c9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518100980 0x7f4518198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451c9ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518101b70 0x7f4518198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451c9ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518102510 0x7f451819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451c9ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45181a03b0 con 0x7f4518102510 2023-08-11T15:44:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451c9ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45181a04f0 con 0x7f4518101b70 2023-08-11T15:44:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451c9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45181a0680 con 0x7f4518100980 2023-08-11T15:44:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518100980 0x7f4518198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f4516d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518102510 0x7f451819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518100980 0x7f4518198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:54946/0 (socket says 172.21.15.32:54946) 2023-08-11T15:44:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f451659c700 1 -- 172.21.15.32:0/3111002686 learned_addr learned my addr 172.21.15.32:0/3111002686 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.390+0000 7f450ffff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518101b70 0x7f4518198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f451659c700 1 -- 172.21.15.32:0/3111002686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518101b70 msgr2=0x7f4518198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f451659c700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518101b70 0x7f4518198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f451659c700 1 -- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518102510 msgr2=0x7f451819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f451659c700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518102510 0x7f451819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f451659c700 1 -- 172.21.15.32:0/3111002686 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080095f0 con 0x7f4518100980 2023-08-11T15:44:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f4516d9d700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518102510 0x7f451819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f451659c700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518100980 0x7f4518198830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4508009fd0 tx=0x7f450800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f450f7fe700 1 -- 172.21.15.32:0/3111002686 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450801d070 con 0x7f4518100980 2023-08-11T15:44:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45181a0900 con 0x7f4518100980 2023-08-11T15:44:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f450f7fe700 1 -- 172.21.15.32:0/3111002686 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4508005050 con 0x7f4518100980 2023-08-11T15:44:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.391+0000 7f450f7fe700 1 -- 172.21.15.32:0/3111002686 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508004a70 con 0x7f4518100980 2023-08-11T15:44:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.392+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45181a0d80 con 0x7f4518100980 2023-08-11T15:44:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.392+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451810c170 con 0x7f4518100980 2023-08-11T15:44:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.394+0000 7f450f7fe700 1 -- 172.21.15.32:0/3111002686 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f4508005450 con 0x7f4518100980 2023-08-11T15:44:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.396+0000 7f450f7fe700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f450409ffd0 0x7f45040a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.396+0000 7f450f7fe700 1 -- 172.21.15.32:0/3111002686 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45080cc5a0 con 0x7f4518100980 2023-08-11T15:44:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.396+0000 7f450ffff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f450409ffd0 0x7f45040a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.397+0000 7f450ffff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f450409ffd0 0x7f45040a2490 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4500009c60 tx=0x7f4500009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.398+0000 7f450f7fe700 1 -- 172.21.15.32:0/3111002686 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4508096170 con 0x7f4518100980 2023-08-11T15:44:49.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:49 smithi134 ceph-mon[130447]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:49 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/767744541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:49.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:49 smithi032 ceph-mon[97103]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:49.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:49 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/767744541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:49.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:49 smithi192 ceph-mon[130806]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-11T15:44:49.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:49 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/767744541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:49.659+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f451810c330 con 0x7f4518100980 2023-08-11T15:44:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.161+0000 7f450f7fe700 1 -- 172.21.15.32:0/3111002686 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4508099790 con 0x7f4518100980 2023-08-11T15:44:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f450409ffd0 msgr2=0x7f45040a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f450409ffd0 0x7f45040a2490 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4500009c60 tx=0x7f4500009260 comp rx=0 tx=0).stop 2023-08-11T15:44:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518100980 msgr2=0x7f4518198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518100980 0x7f4518198830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4508009fd0 tx=0x7f450800f690 comp rx=0 tx=0).stop 2023-08-11T15:44:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 shutdown_connections 2023-08-11T15:44:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f450409ffd0 0x7f45040a2490 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518102510 0x7f451819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4518100980 0x7f4518198830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 --2- 172.21.15.32:0/3111002686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4518101b70 0x7f4518198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.164+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 >> 172.21.15.32:0/3111002686 conn(0x7f45180fbef0 msgr2=0x7f45180fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.165+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 shutdown_connections 2023-08-11T15:44:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.165+0000 7f451c9ff700 1 -- 172.21.15.32:0/3111002686 wait complete. 2023-08-11T15:44:50.187 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-08-11T15:44:50.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-08-11T15:44:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 -- 172.21.15.32:0/1189460702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 msgr2=0x7fe168073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/1189460702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168073800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009990 tx=0x7fe15c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 -- 172.21.15.32:0/1189460702 shutdown_connections 2023-08-11T15:44:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/1189460702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1681087f0 0x7fe16810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/1189460702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168074960 0x7fe168072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/1189460702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168073800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 -- 172.21.15.32:0/1189460702 >> 172.21.15.32:0/1189460702 conn(0x7fe1680fbea0 msgr2=0x7fe1680fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 -- 172.21.15.32:0/1189460702 shutdown_connections 2023-08-11T15:44:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.394+0000 7fe16d1ff700 1 -- 172.21.15.32:0/1189460702 wait complete. 2023-08-11T15:44:50.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.395+0000 7fe16d1ff700 1 Processor -- start 2023-08-11T15:44:50.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.395+0000 7fe16d1ff700 1 -- start start 2023-08-11T15:44:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16d1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16d1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168074960 0x7fe168198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16d1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1681087f0 0x7fe16819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16d1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1681a03f0 con 0x7fe168073380 2023-08-11T15:44:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16d1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1681a0530 con 0x7fe1681087f0 2023-08-11T15:44:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16d1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1681a06c0 con 0x7fe168074960 2023-08-11T15:44:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40180/0 (socket says 172.21.15.32:40180) 2023-08-11T15:44:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 -- 172.21.15.32:0/4076978248 learned_addr learned my addr 172.21.15.32:0/4076978248 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16759e700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1681087f0 0x7fe16819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe16659c700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168074960 0x7fe168198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 -- 172.21.15.32:0/4076978248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168074960 msgr2=0x7fe168198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168074960 0x7fe168198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 -- 172.21.15.32:0/4076978248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1681087f0 msgr2=0x7fe16819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1681087f0 0x7fe16819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.396+0000 7fe166d9d700 1 -- 172.21.15.32:0/4076978248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0095f0 con 0x7fe168073380 2023-08-11T15:44:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.397+0000 7fe166d9d700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168198870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e9a0 tx=0x7fe15800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.397+0000 7fe14ffff700 1 -- 172.21.15.32:0/4076978248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158009790 con 0x7fe168073380 2023-08-11T15:44:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.397+0000 7fe14ffff700 1 -- 172.21.15.32:0/4076978248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe158010430 con 0x7fe168073380 2023-08-11T15:44:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.397+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1681a09a0 con 0x7fe168073380 2023-08-11T15:44:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.397+0000 7fe14ffff700 1 -- 172.21.15.32:0/4076978248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158018680 con 0x7fe168073380 2023-08-11T15:44:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.397+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1681a0eb0 con 0x7fe168073380 2023-08-11T15:44:50.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.398+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe168106ea0 con 0x7fe168073380 2023-08-11T15:44:50.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.400+0000 7fe14ffff700 1 -- 172.21.15.32:0/4076978248 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fe1580105a0 con 0x7fe168073380 2023-08-11T15:44:50.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.401+0000 7fe14ffff700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe15009ff50 0x7fe1500a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:50.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.402+0000 7fe14ffff700 1 -- 172.21.15.32:0/4076978248 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe158014070 con 0x7fe168073380 2023-08-11T15:44:50.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.402+0000 7fe16659c700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe15009ff50 0x7fe1500a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:50.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.402+0000 7fe16659c700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe15009ff50 0x7fe1500a2410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00b2e0 tx=0x7fe15c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:50.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.405+0000 7fe14ffff700 1 -- 172.21.15.32:0/4076978248 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe158051c30 con 0x7fe168073380 2023-08-11T15:44:50.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:50 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1330955825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:50.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:50 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3111002686' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T15:44:50.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:50 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T15:44:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:50 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1330955825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:50 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3111002686' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T15:44:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:50 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T15:44:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:50 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1330955825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:50 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3111002686' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T15:44:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:50 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T15:44:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:50.663+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe16810e370 con 0x7fe168073380 2023-08-11T15:44:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.168+0000 7fe14ffff700 1 -- 172.21.15.32:0/4076978248 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe16810e370 con 0x7fe168073380 2023-08-11T15:44:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe15009ff50 msgr2=0x7fe1500a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe15009ff50 0x7fe1500a2410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00b2e0 tx=0x7fe15c005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 msgr2=0x7fe168198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168198870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e9a0 tx=0x7fe15800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 shutdown_connections 2023-08-11T15:44:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe15009ff50 0x7fe1500a2410 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe168073380 0x7fe168198870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1681087f0 0x7fe16819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 --2- 172.21.15.32:0/4076978248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168074960 0x7fe168198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 >> 172.21.15.32:0/4076978248 conn(0x7fe1680fbea0 msgr2=0x7fe1680fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 shutdown_connections 2023-08-11T15:44:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.172+0000 7fe16d1ff700 1 -- 172.21.15.32:0/4076978248 wait complete. 2023-08-11T15:44:51.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-08-11T15:44:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 -- 172.21.15.32:0/639487153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 msgr2=0x7f6438108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 --2- 172.21.15.32:0/639487153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f6438108d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f643000b0d0 tx=0x7f643000b3e0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 -- 172.21.15.32:0/639487153 shutdown_connections 2023-08-11T15:44:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 --2- 172.21.15.32:0/639487153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f6438108d50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 --2- 172.21.15.32:0/639487153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438100910 0x7f6438100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 --2- 172.21.15.32:0/639487153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438101b00 0x7f6438101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 -- 172.21.15.32:0/639487153 >> 172.21.15.32:0/639487153 conn(0x7f64380fbe60 msgr2=0x7f64380fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 -- 172.21.15.32:0/639487153 shutdown_connections 2023-08-11T15:44:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.408+0000 7f64401ff700 1 -- 172.21.15.32:0/639487153 wait complete. 2023-08-11T15:44:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.409+0000 7f64401ff700 1 Processor -- start 2023-08-11T15:44:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f64401ff700 1 -- start start 2023-08-11T15:44:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f64401ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438100910 0x7f64381966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f64401ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438101b00 0x7f6438196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f64401ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f643819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f64401ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643819e230 con 0x7f64381024a0 2023-08-11T15:44:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f64401ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643819e370 con 0x7f6438101b00 2023-08-11T15:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f64401ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643819e500 con 0x7f6438100910 2023-08-11T15:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f643e79c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f643819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f643df9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438100910 0x7f64381966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.410+0000 7f643e79c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f643819dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40202/0 (socket says 172.21.15.32:40202) 2023-08-11T15:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643e79c700 1 -- 172.21.15.32:0/3133780637 learned_addr learned my addr 172.21.15.32:0/3133780637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643d79a700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438101b00 0x7f6438196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643e79c700 1 -- 172.21.15.32:0/3133780637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438100910 msgr2=0x7f64381966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643e79c700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438100910 0x7f64381966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643e79c700 1 -- 172.21.15.32:0/3133780637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438101b00 msgr2=0x7f6438196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643e79c700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438101b00 0x7f6438196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643e79c700 1 -- 172.21.15.32:0/3133780637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430009d00 con 0x7f64381024a0 2023-08-11T15:44:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f643e79c700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f643819dc60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6430008ec0 tx=0x7f6430008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f642effd700 1 -- 172.21.15.32:0/3133780637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000e040 con 0x7f64381024a0 2023-08-11T15:44:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643819e780 con 0x7f64381024a0 2023-08-11T15:44:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f642effd700 1 -- 172.21.15.32:0/3133780637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6430005070 con 0x7f64381024a0 2023-08-11T15:44:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.411+0000 7f642effd700 1 -- 172.21.15.32:0/3133780637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643001b6c0 con 0x7f64381024a0 2023-08-11T15:44:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.412+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643819eba0 con 0x7f64381024a0 2023-08-11T15:44:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.413+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6438107e00 con 0x7f64381024a0 2023-08-11T15:44:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.415+0000 7f642effd700 1 -- 172.21.15.32:0/3133780637 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f64300051e0 con 0x7f64381024a0 2023-08-11T15:44:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.417+0000 7f642effd700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f642409ff50 0x7f64240a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.417+0000 7f642effd700 1 -- 172.21.15.32:0/3133780637 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f64300cc490 con 0x7f64381024a0 2023-08-11T15:44:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.417+0000 7f643df9b700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f642409ff50 0x7f64240a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.418+0000 7f643df9b700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f642409ff50 0x7f64240a2410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6438101960 tx=0x7f6434009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.421+0000 7f642effd700 1 -- 172.21.15.32:0/3133780637 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f64300960c0 con 0x7f64381024a0 2023-08-11T15:44:51.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:51 smithi134 ceph-mon[130447]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-08-11T15:44:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:51 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-11T15:44:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:51 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:51 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4076978248' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T15:44:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:51 smithi032 ceph-mon[97103]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-08-11T15:44:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:51 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-11T15:44:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:51 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:51.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:51 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4076978248' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T15:44:51.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:51 smithi192 ceph-mon[130806]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-08-11T15:44:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:51 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-11T15:44:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:51 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:51.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:51 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4076978248' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T15:44:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:51.677+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f643802cc20 con 0x7f64381024a0 2023-08-11T15:44:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.178+0000 7f642effd700 1 -- 172.21.15.32:0/3133780637 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f64300996e0 con 0x7f64381024a0 2023-08-11T15:44:52.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f642409ff50 msgr2=0x7f64240a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f642409ff50 0x7f64240a2410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6438101960 tx=0x7f6434009260 comp rx=0 tx=0).stop 2023-08-11T15:44:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 msgr2=0x7f643819dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f643819dc60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6430008ec0 tx=0x7f6430008ef0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 shutdown_connections 2023-08-11T15:44:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f642409ff50 0x7f64240a2410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64381024a0 0x7f643819dc60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438100910 0x7f64381966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 --2- 172.21.15.32:0/3133780637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438101b00 0x7f6438196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 >> 172.21.15.32:0/3133780637 conn(0x7f64380fbe60 msgr2=0x7f6438104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 shutdown_connections 2023-08-11T15:44:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.181+0000 7f64401ff700 1 -- 172.21.15.32:0/3133780637 wait complete. 2023-08-11T15:44:52.215 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-08-11T15:44:52.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-08-11T15:44:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 -- 172.21.15.32:0/1309252251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4100910 msgr2=0x7f50d4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 --2- 172.21.15.32:0/1309252251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4100910 0x7f50d4100d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50c40099c0 tx=0x7f50c4009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 -- 172.21.15.32:0/1309252251 shutdown_connections 2023-08-11T15:44:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 --2- 172.21.15.32:0/1309252251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d4100910 0x7f50d4100d30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 --2- 172.21.15.32:0/1309252251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50d41024a0 0x7f50d4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 --2- 172.21.15.32:0/1309252251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50d4101b00 0x7f50d4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 -- 172.21.15.32:0/1309252251 >> 172.21.15.32:0/1309252251 conn(0x7f50d40fbe60 msgr2=0x7f50d40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.432+0000 7f50d97ff700 1 -- 172.21.15.32:0/1309252251 shutdown_connections 2023-08-11T15:44:52.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.433+0000 7f50d97ff700 1 -- 172.21.15.32:0/1309252251 wait complete. 2023-08-11T15:44:52.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.433+0000 7f50d97ff700 1 Processor -- start 2023-08-11T15:44:52.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d97ff700 1 -- start start 2023-08-11T15:44:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d97ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50d4100910 0x7f50d40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d97ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50d4101b00 0x7f50d4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d97ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d41024a0 0x7f50d4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d97ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40739f0 con 0x7f50d41024a0 2023-08-11T15:44:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d97ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d4073b60 con 0x7f50d4100910 2023-08-11T15:44:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d97ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d41a0690 con 0x7f50d4101b00 2023-08-11T15:44:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d41024a0 0x7f50d4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d41024a0 0x7f50d4073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40220/0 (socket says 172.21.15.32:40220) 2023-08-11T15:44:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d37fe700 1 -- 172.21.15.32:0/3144439545 learned_addr learned my addr 172.21.15.32:0/3144439545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d27fc700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50d4101b00 0x7f50d4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.434+0000 7f50d2ffd700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50d4100910 0x7f50d40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d37fe700 1 -- 172.21.15.32:0/3144439545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50d4101b00 msgr2=0x7f50d4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d37fe700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50d4101b00 0x7f50d4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d37fe700 1 -- 172.21.15.32:0/3144439545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50d4100910 msgr2=0x7f50d40746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d37fe700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50d4100910 0x7f50d40746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d37fe700 1 -- 172.21.15.32:0/3144439545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40095f0 con 0x7f50d41024a0 2023-08-11T15:44:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d37fe700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d41024a0 0x7f50d4073250 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00ef30 tx=0x7f50cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50bbfff700 1 -- 172.21.15.32:0/3144439545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc018070 con 0x7f50d41024a0 2023-08-11T15:44:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d41a0910 con 0x7f50d41024a0 2023-08-11T15:44:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50bbfff700 1 -- 172.21.15.32:0/3144439545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50cc00f040 con 0x7f50d41024a0 2023-08-11T15:44:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50bbfff700 1 -- 172.21.15.32:0/3144439545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00cd00 con 0x7f50d41024a0 2023-08-11T15:44:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.435+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d41a0e70 con 0x7f50d41024a0 2023-08-11T15:44:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.436+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d4107890 con 0x7f50d41024a0 2023-08-11T15:44:52.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.438+0000 7f50bbfff700 1 -- 172.21.15.32:0/3144439545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f50cc003a00 con 0x7f50d41024a0 2023-08-11T15:44:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.440+0000 7f50bbfff700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f50bc09ff50 0x7f50bc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.440+0000 7f50bbfff700 1 -- 172.21.15.32:0/3144439545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f50cc0caf80 con 0x7f50d41024a0 2023-08-11T15:44:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.440+0000 7f50d2ffd700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f50bc09ff50 0x7f50bc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.441+0000 7f50d2ffd700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f50bc09ff50 0x7f50bc0a2410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009990 tx=0x7f50c4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.444+0000 7f50bbfff700 1 -- 172.21.15.32:0/3144439545 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f50cc094b50 con 0x7f50d41024a0 2023-08-11T15:44:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:52 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4076978248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-11T15:44:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:52 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:52 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:44:52.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:52 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3133780637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-11T15:44:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4076978248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-11T15:44:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:44:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3133780637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-11T15:44:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:52 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4076978248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-11T15:44:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:52 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:44:52.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:52 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:44:52.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:52 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3133780637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-11T15:44:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.689+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f50d402cdc0 con 0x7f50bc09ff50 2023-08-11T15:44:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.731+0000 7f50bbfff700 1 -- 172.21.15.32:0/3144439545 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f50d402cdc0 con 0x7f50bc09ff50 2023-08-11T15:44:52.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f50bc09ff50 msgr2=0x7f50bc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f50bc09ff50 0x7f50bc0a2410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009990 tx=0x7f50c4005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d41024a0 msgr2=0x7f50d4073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d41024a0 0x7f50d4073250 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00ef30 tx=0x7f50cc00c3a0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 shutdown_connections 2023-08-11T15:44:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f50bc09ff50 0x7f50bc0a2410 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50d41024a0 0x7f50d4073250 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50d4100910 0x7f50d40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 --2- 172.21.15.32:0/3144439545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50d4101b00 0x7f50d4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 >> 172.21.15.32:0/3144439545 conn(0x7f50d40fbe60 msgr2=0x7f50d40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:52.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 shutdown_connections 2023-08-11T15:44:52.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.733+0000 7f50d97ff700 1 -- 172.21.15.32:0/3144439545 wait complete. 2023-08-11T15:44:52.756 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-08-11T15:44:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 -- 172.21.15.32:0/2298305857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68100910 msgr2=0x7fdf68100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/2298305857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68100910 0x7fdf68100d30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf640099c0 tx=0x7fdf64009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 -- 172.21.15.32:0/2298305857 shutdown_connections 2023-08-11T15:44:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/2298305857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68100910 0x7fdf68100d30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/2298305857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf681024a0 0x7fdf68108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/2298305857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf68101b00 0x7fdf68101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 -- 172.21.15.32:0/2298305857 >> 172.21.15.32:0/2298305857 conn(0x7fdf680fbe60 msgr2=0x7fdf680fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 -- 172.21.15.32:0/2298305857 shutdown_connections 2023-08-11T15:44:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.962+0000 7fdf6fdff700 1 -- 172.21.15.32:0/2298305857 wait complete. 2023-08-11T15:44:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 Processor -- start 2023-08-11T15:44:52.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 -- start start 2023-08-11T15:44:52.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf68100910 0x7fdf68198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68101b00 0x7fdf68198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf681024a0 0x7fdf6819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681a0410 con 0x7fdf68101b00 2023-08-11T15:44:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681a0550 con 0x7fdf681024a0 2023-08-11T15:44:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.963+0000 7fdf6fdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681a06e0 con 0x7fdf68100910 2023-08-11T15:44:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68101b00 0x7fdf68198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68101b00 0x7fdf68198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40248/0 (socket says 172.21.15.32:40248) 2023-08-11T15:44:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 -- 172.21.15.32:0/5889376 learned_addr learned my addr 172.21.15.32:0/5889376 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6db9b700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf68100910 0x7fdf68198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6e39c700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf681024a0 0x7fdf6819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 -- 172.21.15.32:0/5889376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf68100910 msgr2=0x7fdf68198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf68100910 0x7fdf68198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 -- 172.21.15.32:0/5889376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf681024a0 msgr2=0x7fdf6819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf681024a0 0x7fdf6819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 -- 172.21.15.32:0/5889376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640095f0 con 0x7fdf68101b00 2023-08-11T15:44:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.964+0000 7fdf6d39a700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68101b00 0x7fdf68198dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800e9a0 tx=0x7fdf5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.965+0000 7fdf5effd700 1 -- 172.21.15.32:0/5889376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58009790 con 0x7fdf68101b00 2023-08-11T15:44:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.965+0000 7fdf5effd700 1 -- 172.21.15.32:0/5889376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf58010430 con 0x7fdf68101b00 2023-08-11T15:44:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.965+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf681a09c0 con 0x7fdf68101b00 2023-08-11T15:44:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.965+0000 7fdf5effd700 1 -- 172.21.15.32:0/5889376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58018680 con 0x7fdf68101b00 2023-08-11T15:44:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.965+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681a0ed0 con 0x7fdf68101b00 2023-08-11T15:44:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.966+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf68192f60 con 0x7fdf68101b00 2023-08-11T15:44:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.968+0000 7fdf5effd700 1 -- 172.21.15.32:0/5889376 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fdf580105a0 con 0x7fdf68101b00 2023-08-11T15:44:52.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.969+0000 7fdf5effd700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdf5409ff50 0x7fdf540a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:52.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.970+0000 7fdf5effd700 1 -- 172.21.15.32:0/5889376 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdf58014070 con 0x7fdf68101b00 2023-08-11T15:44:52.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.970+0000 7fdf6db9b700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdf5409ff50 0x7fdf540a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:52.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.970+0000 7fdf6db9b700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdf5409ff50 0x7fdf540a2410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009990 tx=0x7fdf64005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:52.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:52.973+0000 7fdf5effd700 1 -- 172.21.15.32:0/5889376 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fdf58051c30 con 0x7fdf68101b00 2023-08-11T15:44:53.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:44:52.696+0000 7f60ca023700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:44:53.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:44:52.696+0000 7f60ca023700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:44:53.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:44:52.696+0000 7f60ca023700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:44:53.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:44:52.696+0000 7f60ca023700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:44:53.054 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:44:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:44:52.696+0000 7f60ca023700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:44:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.222+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf6802ce30 con 0x7fdf5409ff50 2023-08-11T15:44:53.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.227+0000 7fdf5effd700 1 -- 172.21.15.32:0/5889376 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fdf6802ce30 con 0x7fdf5409ff50 2023-08-11T15:44:53.231 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/_nogroup/sv_0/d4bfd765-c3c5-4a7b-bd64-4675033ff24f 2023-08-11T15:44:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.229+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdf5409ff50 msgr2=0x7fdf540a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.229+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdf5409ff50 0x7fdf540a2410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009990 tx=0x7fdf64005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68101b00 msgr2=0x7fdf68198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68101b00 0x7fdf68198dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800e9a0 tx=0x7fdf5800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 shutdown_connections 2023-08-11T15:44:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fdf5409ff50 0x7fdf540a2410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf681024a0 0x7fdf6819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf68100910 0x7fdf68198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 --2- 172.21.15.32:0/5889376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf68101b00 0x7fdf68198dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 >> 172.21.15.32:0/5889376 conn(0x7fdf680fbe60 msgr2=0x7fdf68107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 shutdown_connections 2023-08-11T15:44:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.230+0000 7fdf6fdff700 1 -- 172.21.15.32:0/5889376 wait complete. 2023-08-11T15:44:53.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-08-11T15:44:53.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.463+0000 7f4a261ff700 1 -- 172.21.15.32:0/861969316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a20100910 msgr2=0x7f4a20100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.463+0000 7f4a261ff700 1 --2- 172.21.15.32:0/861969316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a20100910 0x7f4a20100d30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a100099c0 tx=0x7f4a10009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.464+0000 7f4a261ff700 1 -- 172.21.15.32:0/861969316 shutdown_connections 2023-08-11T15:44:53.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.464+0000 7f4a261ff700 1 --2- 172.21.15.32:0/861969316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a20100910 0x7f4a20100d30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.464+0000 7f4a261ff700 1 --2- 172.21.15.32:0/861969316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a201024a0 0x7f4a20108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.464+0000 7f4a261ff700 1 --2- 172.21.15.32:0/861969316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a20101b00 0x7f4a20101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.464+0000 7f4a261ff700 1 -- 172.21.15.32:0/861969316 >> 172.21.15.32:0/861969316 conn(0x7f4a200fbe60 msgr2=0x7f4a200fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.464+0000 7f4a261ff700 1 -- 172.21.15.32:0/861969316 shutdown_connections 2023-08-11T15:44:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.464+0000 7f4a261ff700 1 -- 172.21.15.32:0/861969316 wait complete. 2023-08-11T15:44:53.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 Processor -- start 2023-08-11T15:44:53.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 -- start start 2023-08-11T15:44:53.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a20100910 0x7f4a20198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a20101b00 0x7f4a20198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a201024a0 0x7f4a2019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a201a03e0 con 0x7f4a201024a0 2023-08-11T15:44:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a201a0520 con 0x7f4a20101b00 2023-08-11T15:44:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a261ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a201a06b0 con 0x7f4a20100910 2023-08-11T15:44:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a1ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a201024a0 0x7f4a2019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a1ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a201024a0 0x7f4a2019fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40280/0 (socket says 172.21.15.32:40280) 2023-08-11T15:44:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.465+0000 7f4a1ffff700 1 -- 172.21.15.32:0/4062405003 learned_addr learned my addr 172.21.15.32:0/4062405003 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1effd700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a20101b00 0x7f4a20198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1f7fe700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a20100910 0x7f4a20198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1ffff700 1 -- 172.21.15.32:0/4062405003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a20100910 msgr2=0x7f4a20198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1ffff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a20100910 0x7f4a20198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1ffff700 1 -- 172.21.15.32:0/4062405003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a20101b00 msgr2=0x7f4a20198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1ffff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a20101b00 0x7f4a20198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1ffff700 1 -- 172.21.15.32:0/4062405003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a100095f0 con 0x7f4a201024a0 2023-08-11T15:44:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1ffff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a201024a0 0x7f4a2019fe10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a18007ee0 tx=0x7f4a1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1cff9700 1 -- 172.21.15.32:0/4062405003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18017070 con 0x7f4a201024a0 2023-08-11T15:44:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1cff9700 1 -- 172.21.15.32:0/4062405003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a1800f040 con 0x7f4a201024a0 2023-08-11T15:44:53.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a201a0990 con 0x7f4a201024a0 2023-08-11T15:44:53.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.466+0000 7f4a1cff9700 1 -- 172.21.15.32:0/4062405003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18005580 con 0x7f4a201024a0 2023-08-11T15:44:53.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.467+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a201a0e50 con 0x7f4a201024a0 2023-08-11T15:44:53.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.467+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2010c170 con 0x7f4a201024a0 2023-08-11T15:44:53.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:53 smithi134 ceph-mon[130447]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2023-08-11T15:44:53.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:53 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3133780637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-11T15:44:53.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:53 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:44:53.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.471+0000 7f4a1cff9700 1 -- 172.21.15.32:0/4062405003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f4a1800db20 con 0x7f4a201024a0 2023-08-11T15:44:53.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.473+0000 7f4a1cff9700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a0809ff50 0x7f4a080a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.473+0000 7f4a1cff9700 1 -- 172.21.15.32:0/4062405003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4a180cb260 con 0x7f4a201024a0 2023-08-11T15:44:53.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.473+0000 7f4a1f7fe700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a0809ff50 0x7f4a080a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.474+0000 7f4a1f7fe700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a0809ff50 0x7f4a080a2410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009990 tx=0x7f4a10005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:53.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.475+0000 7f4a1cff9700 1 -- 172.21.15.32:0/4062405003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4a18094da0 con 0x7f4a201024a0 2023-08-11T15:44:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:53 smithi032 ceph-mon[97103]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2023-08-11T15:44:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:53 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3133780637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-11T15:44:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:53 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:44:53.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:53 smithi192 ceph-mon[130806]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2023-08-11T15:44:53.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:53 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3133780637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-11T15:44:53.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:53 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:44:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.717+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a201084a0 con 0x7f4a0809ff50 2023-08-11T15:44:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.743+0000 7f4a1cff9700 1 -- 172.21.15.32:0/4062405003 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4a201084a0 con 0x7f4a0809ff50 2023-08-11T15:44:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.745+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a0809ff50 msgr2=0x7f4a080a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.745+0000 7f4a261ff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a0809ff50 0x7f4a080a2410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009990 tx=0x7f4a10005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.745+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a201024a0 msgr2=0x7f4a2019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.745+0000 7f4a261ff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a201024a0 0x7f4a2019fe10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a18007ee0 tx=0x7f4a1800d3b0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 shutdown_connections 2023-08-11T15:44:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a0809ff50 0x7f4a080a2410 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a201024a0 0x7f4a2019fe10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a20100910 0x7f4a20198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 --2- 172.21.15.32:0/4062405003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a20101b00 0x7f4a20198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 >> 172.21.15.32:0/4062405003 conn(0x7f4a200fbe60 msgr2=0x7f4a200fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 shutdown_connections 2023-08-11T15:44:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.746+0000 7f4a261ff700 1 -- 172.21.15.32:0/4062405003 wait complete. 2023-08-11T15:44:53.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-08-11T15:44:53.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 -- 172.21.15.32:0/1165166152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648100910 msgr2=0x7fb648100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/1165166152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648100910 0x7fb648100d30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6380099c0 tx=0x7fb638009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 -- 172.21.15.32:0/1165166152 shutdown_connections 2023-08-11T15:44:53.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/1165166152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6481024a0 0x7fb648108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/1165166152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648100910 0x7fb648100d30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/1165166152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648101b00 0x7fb648101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 -- 172.21.15.32:0/1165166152 >> 172.21.15.32:0/1165166152 conn(0x7fb6480fbe60 msgr2=0x7fb6480fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 -- 172.21.15.32:0/1165166152 shutdown_connections 2023-08-11T15:44:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.972+0000 7fb64e1ff700 1 -- 172.21.15.32:0/1165166152 wait complete. 2023-08-11T15:44:53.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.973+0000 7fb64e1ff700 1 Processor -- start 2023-08-11T15:44:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.973+0000 7fb64e1ff700 1 -- start start 2023-08-11T15:44:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.973+0000 7fb64e1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648100910 0x7fb648198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb64e1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648101b00 0x7fb648198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb64e1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6481024a0 0x7fb64819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb64e1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a0410 con 0x7fb648100910 2023-08-11T15:44:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb64e1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a0550 con 0x7fb648101b00 2023-08-11T15:44:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb64e1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a06e0 con 0x7fb6481024a0 2023-08-11T15:44:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648100910 0x7fb648198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648100910 0x7fb648198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40300/0 (socket says 172.21.15.32:40300) 2023-08-11T15:44:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 -- 172.21.15.32:0/2489094884 learned_addr learned my addr 172.21.15.32:0/2489094884 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb646ffd700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648101b00 0x7fb648198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb647fff700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6481024a0 0x7fb64819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 -- 172.21.15.32:0/2489094884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6481024a0 msgr2=0x7fb64819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6481024a0 0x7fb64819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 -- 172.21.15.32:0/2489094884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648101b00 msgr2=0x7fb648198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648101b00 0x7fb648198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.974+0000 7fb6477fe700 1 -- 172.21.15.32:0/2489094884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380095f0 con 0x7fb648100910 2023-08-11T15:44:53.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.975+0000 7fb6477fe700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648100910 0x7fb648198890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098e0 tx=0x7fb63800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:53.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.975+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63801d070 con 0x7fb648100910 2023-08-11T15:44:53.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.975+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb638005050 con 0x7fb648100910 2023-08-11T15:44:53.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.975+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638004b40 con 0x7fb648100910 2023-08-11T15:44:53.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.975+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6481a0960 con 0x7fb648100910 2023-08-11T15:44:53.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.975+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6481a0d30 con 0x7fb648100910 2023-08-11T15:44:53.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.976+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb648192f60 con 0x7fb648100910 2023-08-11T15:44:53.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.978+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fb638004ce0 con 0x7fb648100910 2023-08-11T15:44:53.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.979+0000 7fb644ff9700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb63009ffd0 0x7fb6300a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:53.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.980+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb6380cc250 con 0x7fb648100910 2023-08-11T15:44:53.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.980+0000 7fb646ffd700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb63009ffd0 0x7fb6300a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:53.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.980+0000 7fb646ffd700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb63009ffd0 0x7fb6300a2490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009c60 tx=0x7fb63c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:53.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:53.983+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb638095e20 con 0x7fb648100910 2023-08-11T15:44:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.199+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb6380961f0 con 0x7fb648100910 2023-08-11T15:44:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.255+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb64802ce80 con 0x7fb63009ffd0 2023-08-11T15:44:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.259+0000 7fb644ff9700 1 -- 172.21.15.32:0/2489094884 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fb64802ce80 con 0x7fb63009ffd0 2023-08-11T15:44:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/_nogroup/sv_1/4ac296d7-7135-4c0c-ab47-acc92f9cb432 2023-08-11T15:44:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb63009ffd0 msgr2=0x7fb6300a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb63009ffd0 0x7fb6300a2490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009c60 tx=0x7fb63c009260 comp rx=0 tx=0).stop 2023-08-11T15:44:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648100910 msgr2=0x7fb648198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648100910 0x7fb648198890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098e0 tx=0x7fb63800f690 comp rx=0 tx=0).stop 2023-08-11T15:44:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 shutdown_connections 2023-08-11T15:44:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb63009ffd0 0x7fb6300a2490 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648100910 0x7fb648198890 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6481024a0 0x7fb64819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 --2- 172.21.15.32:0/2489094884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb648101b00 0x7fb648198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 >> 172.21.15.32:0/2489094884 conn(0x7fb6480fbe60 msgr2=0x7fb648107480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 shutdown_connections 2023-08-11T15:44:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.262+0000 7fb64e1ff700 1 -- 172.21.15.32:0/2489094884 wait complete. 2023-08-11T15:44:54.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:44:54.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:54.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:54.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-08-11T15:44:54.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-08-11T15:44:54.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-08-11T15:44:54.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-08-11T15:44:54.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-08-11T15:44:54.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-08-11T15:44:54.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-08-11T15:44:54.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]: dispatch 2023-08-11T15:44:54.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-08-11T15:44:54.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:54 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-08-11T15:44:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 -- 172.21.15.32:0/151471100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4100920 msgr2=0x7f30c4100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 --2- 172.21.15.32:0/151471100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4100920 0x7f30c4100d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f30b40099c0 tx=0x7f30b4009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 -- 172.21.15.32:0/151471100 shutdown_connections 2023-08-11T15:44:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 --2- 172.21.15.32:0/151471100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4100920 0x7f30c4100d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 --2- 172.21.15.32:0/151471100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30c41024b0 0x7f30c4108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 --2- 172.21.15.32:0/151471100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4101b10 0x7f30c4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 -- 172.21.15.32:0/151471100 >> 172.21.15.32:0/151471100 conn(0x7f30c40fbe50 msgr2=0x7f30c40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:54.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 -- 172.21.15.32:0/151471100 shutdown_connections 2023-08-11T15:44:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.486+0000 7f30c91ff700 1 -- 172.21.15.32:0/151471100 wait complete. 2023-08-11T15:44:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.487+0000 7f30c91ff700 1 Processor -- start 2023-08-11T15:44:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.487+0000 7f30c91ff700 1 -- start start 2023-08-11T15:44:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c91ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4100920 0x7f30c4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c91ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4101b10 0x7f30c4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c91ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30c41024b0 0x7f30c419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c91ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41a03c0 con 0x7f30c4101b10 2023-08-11T15:44:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c91ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41a0500 con 0x7f30c41024b0 2023-08-11T15:44:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c91ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41a0690 con 0x7f30c4100920 2023-08-11T15:44:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4100920 0x7f30c4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4101b10 0x7f30c4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4100920 0x7f30c4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:55132/0 (socket says 172.21.15.32:55132) 2023-08-11T15:44:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c359e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30c41024b0 0x7f30c419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 -- 172.21.15.32:0/1161164786 learned_addr learned my addr 172.21.15.32:0/1161164786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 -- 172.21.15.32:0/1161164786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30c41024b0 msgr2=0x7f30c419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30c41024b0 0x7f30c419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 -- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4101b10 msgr2=0x7f30c4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4101b10 0x7f30c4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c2d9d700 1 -- 172.21.15.32:0/1161164786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b40095f0 con 0x7f30c4100920 2023-08-11T15:44:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c359e700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30c41024b0 0x7f30c419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.488+0000 7f30c259c700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4101b10 0x7f30c4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.489+0000 7f30c2d9d700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4100920 0x7f30c4198840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009990 tx=0x7f30b400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.489+0000 7f30abfff700 1 -- 172.21.15.32:0/1161164786 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b401d070 con 0x7f30c4100920 2023-08-11T15:44:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.489+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c41a0910 con 0x7f30c4100920 2023-08-11T15:44:54.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.489+0000 7f30abfff700 1 -- 172.21.15.32:0/1161164786 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30b4005030 con 0x7f30c4100920 2023-08-11T15:44:54.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.489+0000 7f30abfff700 1 -- 172.21.15.32:0/1161164786 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4004b30 con 0x7f30c4100920 2023-08-11T15:44:54.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.489+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c41a0d90 con 0x7f30c4100920 2023-08-11T15:44:54.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.490+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c410c180 con 0x7f30c4100920 2023-08-11T15:44:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.492+0000 7f30abfff700 1 -- 172.21.15.32:0/1161164786 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f30b4005410 con 0x7f30c4100920 2023-08-11T15:44:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.493+0000 7f30abfff700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f30ac0a00a0 0x7f30ac0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.494+0000 7f30abfff700 1 -- 172.21.15.32:0/1161164786 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f30b40cc8a0 con 0x7f30c4100920 2023-08-11T15:44:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.494+0000 7f30c259c700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f30ac0a00a0 0x7f30ac0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.494+0000 7f30c259c700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f30ac0a00a0 0x7f30ac0a2560 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30b8011490 tx=0x7f30b8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.496+0000 7f30abfff700 1 -- 172.21.15.32:0/1161164786 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f30b40961f0 con 0x7f30c4100920 2023-08-11T15:44:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-08-11T15:44:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-08-11T15:44:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-08-11T15:44:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-08-11T15:44:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-08-11T15:44:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-08-11T15:44:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-08-11T15:44:54.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]: dispatch 2023-08-11T15:44:54.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-08-11T15:44:54.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:54 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-08-11T15:44:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:54.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:54.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-08-11T15:44:54.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-08-11T15:44:54.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-08-11T15:44:54.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-08-11T15:44:54.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-08-11T15:44:54.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-08-11T15:44:54.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-08-11T15:44:54.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]: dispatch 2023-08-11T15:44:54.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-08-11T15:44:54.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:54 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-08-11T15:44:54.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.752+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30c410c3c0 con 0x7f30c4100920 2023-08-11T15:44:54.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.754+0000 7f30abfff700 1 -- 172.21.15.32:0/1161164786 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f30b4099810 con 0x7f30c4100920 2023-08-11T15:44:54.756 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:54.756 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:52.172791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:44:54.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.755+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f30ac0a00a0 msgr2=0x7f30ac0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.755+0000 7f30c91ff700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f30ac0a00a0 0x7f30ac0a2560 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30b8011490 tx=0x7f30b8009260 comp rx=0 tx=0).stop 2023-08-11T15:44:54.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.755+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4100920 msgr2=0x7f30c4198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.755+0000 7f30c91ff700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4100920 0x7f30c4198840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009990 tx=0x7f30b400f690 comp rx=0 tx=0).stop 2023-08-11T15:44:54.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 shutdown_connections 2023-08-11T15:44:54.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f30ac0a00a0 0x7f30ac0a2560 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30c41024b0 0x7f30c419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c4100920 0x7f30c4198840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 --2- 172.21.15.32:0/1161164786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c4101b10 0x7f30c4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 >> 172.21.15.32:0/1161164786 conn(0x7f30c40fbe50 msgr2=0x7f30c40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:54.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 shutdown_connections 2023-08-11T15:44:54.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.756+0000 7f30c91ff700 1 -- 172.21.15.32:0/1161164786 wait complete. 2023-08-11T15:44:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-08-11T15:44:54.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:44:54.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.978+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3096558747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78101b30 msgr2=0x7f4a78101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3096558747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78101b30 0x7f4a78101f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0099c0 tx=0x7f4a6c009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3096558747 shutdown_connections 2023-08-11T15:44:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3096558747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 0x7f4a78108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3096558747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a78100940 0x7f4a78100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3096558747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78101b30 0x7f4a78101f90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3096558747 >> 172.21.15.32:0/3096558747 conn(0x7f4a780fbe90 msgr2=0x7f4a780fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3096558747 shutdown_connections 2023-08-11T15:44:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.979+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3096558747 wait complete. 2023-08-11T15:44:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 Processor -- start 2023-08-11T15:44:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 -- start start 2023-08-11T15:44:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78100940 0x7f4a78198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a78101b30 0x7f4a78198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 0x7f4a7819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781a0380 con 0x7f4a781024d0 2023-08-11T15:44:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781a04c0 con 0x7f4a78100940 2023-08-11T15:44:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a7e1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781a0650 con 0x7f4a78101b30 2023-08-11T15:44:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 0x7f4a7819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.980+0000 7f4a777fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78100940 0x7f4a78198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78100940 0x7f4a78198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:36030/0 (socket says 172.21.15.32:36030) 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 0x7f4a7819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47030/0 (socket says 172.21.15.32:47030) 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 -- 172.21.15.32:0/3078516862 learned_addr learned my addr 172.21.15.32:0/3078516862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a76ffd700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a78101b30 0x7f4a78198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 -- 172.21.15.32:0/3078516862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a78101b30 msgr2=0x7f4a78198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a78101b30 0x7f4a78198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 -- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 msgr2=0x7f4a7819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 0x7f4a7819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 -- 172.21.15.32:0/3078516862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0095f0 con 0x7f4a78100940 2023-08-11T15:44:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a77fff700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 0x7f4a7819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.981+0000 7f4a777fe700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78100940 0x7f4a78198800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800e970 tx=0x7f4a6800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.982+0000 7f4a74ff9700 1 -- 172.21.15.32:0/3078516862 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68009790 con 0x7f4a78100940 2023-08-11T15:44:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.982+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a781a0930 con 0x7f4a78100940 2023-08-11T15:44:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.982+0000 7f4a74ff9700 1 -- 172.21.15.32:0/3078516862 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a6800c7e0 con 0x7f4a78100940 2023-08-11T15:44:54.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.982+0000 7f4a74ff9700 1 -- 172.21.15.32:0/3078516862 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68003c70 con 0x7f4a78100940 2023-08-11T15:44:54.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.982+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a781a0e90 con 0x7f4a78100940 2023-08-11T15:44:54.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.983+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78192f90 con 0x7f4a78100940 2023-08-11T15:44:54.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.985+0000 7f4a74ff9700 1 -- 172.21.15.32:0/3078516862 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f4a6800cc50 con 0x7f4a78100940 2023-08-11T15:44:54.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.987+0000 7f4a74ff9700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a600a0020 0x7f4a600a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:54.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.987+0000 7f4a74ff9700 1 -- 172.21.15.32:0/3078516862 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4a68014070 con 0x7f4a78100940 2023-08-11T15:44:54.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.987+0000 7f4a76ffd700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a600a0020 0x7f4a600a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:54.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.987+0000 7f4a76ffd700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a600a0020 0x7f4a600a24e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00b580 tx=0x7f4a6c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:54.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:54.988+0000 7f4a74ff9700 1 -- 172.21.15.32:0/3078516862 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4a68095320 con 0x7f4a78100940 2023-08-11T15:44:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.212+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a7810c1a0 con 0x7f4a78100940 2023-08-11T15:44:55.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.214+0000 7f4a74ff9700 1 -- 172.21.15.32:0/3078516862 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f4a68098940 con 0x7f4a78100940 2023-08-11T15:44:55.217 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:55.217 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:55.185701+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:44:55.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a600a0020 msgr2=0x7f4a600a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a600a0020 0x7f4a600a24e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00b580 tx=0x7f4a6c005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78100940 msgr2=0x7f4a78198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78100940 0x7f4a78198800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800e970 tx=0x7f4a6800ec80 comp rx=0 tx=0).stop 2023-08-11T15:44:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 shutdown_connections 2023-08-11T15:44:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f4a600a0020 0x7f4a600a24e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a781024d0 0x7f4a7819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a78100940 0x7f4a78198800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 --2- 172.21.15.32:0/3078516862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a78101b30 0x7f4a78198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 >> 172.21.15.32:0/3078516862 conn(0x7f4a780fbe90 msgr2=0x7f4a781073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:55.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 shutdown_connections 2023-08-11T15:44:55.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.217+0000 7f4a7e1ff700 1 -- 172.21.15.32:0/3078516862 wait complete. 2023-08-11T15:44:55.238 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-11T15:44:55.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:44:55.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/1659164431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78100930 msgr2=0x7f3c78100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/1659164431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78100930 0x7f3c78100d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c680099c0 tx=0x7f3c68009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/1659164431 shutdown_connections 2023-08-11T15:44:55.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/1659164431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c78100930 0x7f3c78100d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/1659164431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c781024c0 0x7f3c78108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/1659164431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c78101b20 0x7f3c78101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/1659164431 >> 172.21.15.32:0/1659164431 conn(0x7f3c780fbe60 msgr2=0x7f3c780fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:55.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/1659164431 shutdown_connections 2023-08-11T15:44:55.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.440+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/1659164431 wait complete. 2023-08-11T15:44:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 Processor -- start 2023-08-11T15:44:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 -- start start 2023-08-11T15:44:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78100930 0x7f3c780746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c78101b20 0x7f3c78072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c781024c0 0x7f3c78073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780739f0 con 0x7f3c781024c0 2023-08-11T15:44:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c78073b60 con 0x7f3c78100930 2023-08-11T15:44:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.441+0000 7f3c7e1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c781a0650 con 0x7f3c78101b20 2023-08-11T15:44:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c781024c0 0x7f3c78073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c781024c0 0x7f3c78073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47060/0 (socket says 172.21.15.32:47060) 2023-08-11T15:44:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 -- 172.21.15.32:0/319192171 learned_addr learned my addr 172.21.15.32:0/319192171 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c76ffd700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c78101b20 0x7f3c78072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c777fe700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78100930 0x7f3c780746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 -- 172.21.15.32:0/319192171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c78101b20 msgr2=0x7f3c78072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c78101b20 0x7f3c78072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 -- 172.21.15.32:0/319192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78100930 msgr2=0x7f3c780746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78100930 0x7f3c780746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 -- 172.21.15.32:0/319192171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680095f0 con 0x7f3c781024c0 2023-08-11T15:44:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.442+0000 7f3c77fff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c781024c0 0x7f3c78073250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000ef30 tx=0x7f3c7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.443+0000 7f3c74ff9700 1 -- 172.21.15.32:0/319192171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70018070 con 0x7f3c781024c0 2023-08-11T15:44:55.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.443+0000 7f3c74ff9700 1 -- 172.21.15.32:0/319192171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c7000f040 con 0x7f3c781024c0 2023-08-11T15:44:55.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.443+0000 7f3c74ff9700 1 -- 172.21.15.32:0/319192171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000cd00 con 0x7f3c781024c0 2023-08-11T15:44:55.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.443+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c781a08d0 con 0x7f3c781024c0 2023-08-11T15:44:55.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.443+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c781a0d90 con 0x7f3c781024c0 2023-08-11T15:44:55.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.444+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7810c190 con 0x7f3c781024c0 2023-08-11T15:44:55.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.446+0000 7f3c74ff9700 1 -- 172.21.15.32:0/319192171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f3c70003a00 con 0x7f3c781024c0 2023-08-11T15:44:55.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.447+0000 7f3c74ff9700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c600a00a0 0x7f3c600a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.447+0000 7f3c74ff9700 1 -- 172.21.15.32:0/319192171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c700cb230 con 0x7f3c781024c0 2023-08-11T15:44:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.448+0000 7f3c777fe700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c600a00a0 0x7f3c600a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.448+0000 7f3c777fe700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c600a00a0 0x7f3c600a2560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009990 tx=0x7f3c68005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.452+0000 7f3c74ff9700 1 -- 172.21.15.32:0/319192171 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3c70094b80 con 0x7f3c781024c0 2023-08-11T15:44:55.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:55.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 85 B/s wr, 0 op/s 2023-08-11T15:44:55.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-08-11T15:44:55.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-08-11T15:44:55.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-08-11T15:44:55.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-08-11T15:44:55.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-08-11T15:44:55.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-08-11T15:44:55.483 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-08-11T15:44:55.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]': finished 2023-08-11T15:44:55.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-08-11T15:44:55.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-08-11T15:44:55.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: osdmap e89: 12 total, 12 up, 12 in 2023-08-11T15:44:55.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-08-11T15:44:55.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:55.484 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:55 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1161164786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 85 B/s wr, 0 op/s 2023-08-11T15:44:55.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-08-11T15:44:55.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-08-11T15:44:55.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-08-11T15:44:55.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-08-11T15:44:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-08-11T15:44:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-08-11T15:44:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-08-11T15:44:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]': finished 2023-08-11T15:44:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-08-11T15:44:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-08-11T15:44:55.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: osdmap e89: 12 total, 12 up, 12 in 2023-08-11T15:44:55.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-08-11T15:44:55.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:55.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:55 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1161164786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 85 B/s wr, 0 op/s 2023-08-11T15:44:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-08-11T15:44:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-08-11T15:44:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-08-11T15:44:55.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-08-11T15:44:55.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-08-11T15:44:55.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-08-11T15:44:55.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-08-11T15:44:55.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]': finished 2023-08-11T15:44:55.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-08-11T15:44:55.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-08-11T15:44:55.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: osdmap e89: 12 total, 12 up, 12 in 2023-08-11T15:44:55.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-08-11T15:44:55.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:44:55.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:55 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1161164786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:44:55.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.660+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c78060d60 con 0x7f3c600a00a0 2023-08-11T15:44:55.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.675+0000 7f3c74ff9700 1 -- 172.21.15.32:0/319192171 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422261 (secure 0 0 0) 0x7f3c78060d60 con 0x7f3c600a00a0 2023-08-11T15:44:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:55.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c600a00a0 msgr2=0x7f3c600a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c600a00a0 0x7f3c600a2560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009990 tx=0x7f3c68005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c781024c0 msgr2=0x7f3c78073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c781024c0 0x7f3c78073250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000ef30 tx=0x7f3c7000c3a0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 shutdown_connections 2023-08-11T15:44:55.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3c600a00a0 0x7f3c600a2560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c781024c0 0x7f3c78073250 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c78100930 0x7f3c780746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 --2- 172.21.15.32:0/319192171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c78101b20 0x7f3c78072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 >> 172.21.15.32:0/319192171 conn(0x7f3c780fbe60 msgr2=0x7f3c780fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 shutdown_connections 2023-08-11T15:44:55.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.681+0000 7f3c7e1ff700 1 -- 172.21.15.32:0/319192171 wait complete. 2023-08-11T15:44:55.689 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-08-11T15:44:55.703 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2023-08-11T15:44:55.188161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595131,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":147,"num_read_kb":131,"num_write":231,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337400,"kb_used_data":13148,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604424,"statfs":{"total":1151940427776,"available":1151594930176,"internally_reserved":0,"allocated":13463552,"data_stored":7420815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.115481"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184488+0000","last_change":"2023-08-11T15:44:19.862927+0000","last_active":"2023-08-11T15:44:25.184488+0000","last_peered":"2023-08-11T15:44:25.184488+0000","last_clean":"2023-08-11T15:44:25.184488+0000","last_became_active":"2023-08-11T15:44:19.862626+0000","last_became_peered":"2023-08-11T15:44:19.862626+0000","last_unstale":"2023-08-11T15:44:25.184488+0000","last_undegraded":"2023-08-11T15:44:25.184488+0000","last_fullsized":"2023-08-11T15:44:25.184488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:58:44.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241036+0000","last_change":"2023-08-11T15:44:19.852455+0000","last_active":"2023-08-11T15:44:26.241036+0000","last_peered":"2023-08-11T15:44:26.241036+0000","last_clean":"2023-08-11T15:44:26.241036+0000","last_became_active":"2023-08-11T15:44:19.851897+0000","last_became_peered":"2023-08-11T15:44:19.851897+0000","last_unstale":"2023-08-11T15:44:26.241036+0000","last_undegraded":"2023-08-11T15:44:26.241036+0000","last_fullsized":"2023-08-11T15:44:26.241036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:20.816842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241013+0000","last_change":"2023-08-11T15:44:19.855232+0000","last_active":"2023-08-11T15:44:26.241013+0000","last_peered":"2023-08-11T15:44:26.241013+0000","last_clean":"2023-08-11T15:44:26.241013+0000","last_became_active":"2023-08-11T15:44:19.854940+0000","last_became_peered":"2023-08-11T15:44:19.854940+0000","last_unstale":"2023-08-11T15:44:26.241013+0000","last_undegraded":"2023-08-11T15:44:26.241013+0000","last_fullsized":"2023-08-11T15:44:26.241013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:42:52.881533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196583+0000","last_change":"2023-08-11T15:44:19.856137+0000","last_active":"2023-08-11T15:44:25.196583+0000","last_peered":"2023-08-11T15:44:25.196583+0000","last_clean":"2023-08-11T15:44:25.196583+0000","last_became_active":"2023-08-11T15:44:19.855852+0000","last_became_peered":"2023-08-11T15:44:19.855852+0000","last_unstale":"2023-08-11T15:44:25.196583+0000","last_undegraded":"2023-08-11T15:44:25.196583+0000","last_fullsized":"2023-08-11T15:44:25.196583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:21.362833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194351+0000","last_change":"2023-08-11T15:44:19.862618+0000","last_active":"2023-08-11T15:44:54.194351+0000","last_peered":"2023-08-11T15:44:54.194351+0000","last_clean":"2023-08-11T15:44:54.194351+0000","last_became_active":"2023-08-11T15:44:19.862265+0000","last_became_peered":"2023-08-11T15:44:19.862265+0000","last_unstale":"2023-08-11T15:44:54.194351+0000","last_undegraded":"2023-08-11T15:44:54.194351+0000","last_fullsized":"2023-08-11T15:44:54.194351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:53.783047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909293+0000","last_change":"2023-08-11T15:44:19.857801+0000","last_active":"2023-08-11T15:44:26.909293+0000","last_peered":"2023-08-11T15:44:26.909293+0000","last_clean":"2023-08-11T15:44:26.909293+0000","last_became_active":"2023-08-11T15:44:19.857388+0000","last_became_peered":"2023-08-11T15:44:19.857388+0000","last_unstale":"2023-08-11T15:44:26.909293+0000","last_undegraded":"2023-08-11T15:44:26.909293+0000","last_fullsized":"2023-08-11T15:44:26.909293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:55:45.696116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189336+0000","last_change":"2023-08-11T15:44:19.859334+0000","last_active":"2023-08-11T15:44:54.189336+0000","last_peered":"2023-08-11T15:44:54.189336+0000","last_clean":"2023-08-11T15:44:54.189336+0000","last_became_active":"2023-08-11T15:44:19.859073+0000","last_became_peered":"2023-08-11T15:44:19.859073+0000","last_unstale":"2023-08-11T15:44:54.189336+0000","last_undegraded":"2023-08-11T15:44:54.189336+0000","last_fullsized":"2023-08-11T15:44:54.189336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:37:20.633727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178676+0000","last_change":"2023-08-11T15:44:19.853064+0000","last_active":"2023-08-11T15:44:25.178676+0000","last_peered":"2023-08-11T15:44:25.178676+0000","last_clean":"2023-08-11T15:44:25.178676+0000","last_became_active":"2023-08-11T15:44:19.852736+0000","last_became_peered":"2023-08-11T15:44:19.852736+0000","last_unstale":"2023-08-11T15:44:25.178676+0000","last_undegraded":"2023-08-11T15:44:25.178676+0000","last_fullsized":"2023-08-11T15:44:25.178676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:31.513054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497471+0000","last_change":"2023-08-11T15:44:19.861912+0000","last_active":"2023-08-11T15:44:27.497471+0000","last_peered":"2023-08-11T15:44:27.497471+0000","last_clean":"2023-08-11T15:44:27.497471+0000","last_became_active":"2023-08-11T15:44:19.861374+0000","last_became_peered":"2023-08-11T15:44:19.861374+0000","last_unstale":"2023-08-11T15:44:27.497471+0000","last_undegraded":"2023-08-11T15:44:27.497471+0000","last_fullsized":"2023-08-11T15:44:27.497471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:55.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909786+0000","last_change":"2023-08-11T15:44:19.858862+0000","last_active":"2023-08-11T15:44:26.909786+0000","last_peered":"2023-08-11T15:44:26.909786+0000","last_clean":"2023-08-11T15:44:26.909786+0000","last_became_active":"2023-08-11T15:44:19.858512+0000","last_became_peered":"2023-08-11T15:44:19.858512+0000","last_unstale":"2023-08-11T15:44:26.909786+0000","last_undegraded":"2023-08-11T15:44:26.909786+0000","last_fullsized":"2023-08-11T15:44:26.909786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:53:27.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190057+0000","last_change":"2023-08-11T15:44:19.864063+0000","last_active":"2023-08-11T15:44:54.190057+0000","last_peered":"2023-08-11T15:44:54.190057+0000","last_clean":"2023-08-11T15:44:54.190057+0000","last_became_active":"2023-08-11T15:44:19.863081+0000","last_became_peered":"2023-08-11T15:44:19.863081+0000","last_unstale":"2023-08-11T15:44:54.190057+0000","last_undegraded":"2023-08-11T15:44:54.190057+0000","last_fullsized":"2023-08-11T15:44:54.190057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:19:47.689166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196608+0000","last_change":"2023-08-11T15:44:19.855347+0000","last_active":"2023-08-11T15:44:25.196608+0000","last_peered":"2023-08-11T15:44:25.196608+0000","last_clean":"2023-08-11T15:44:25.196608+0000","last_became_active":"2023-08-11T15:44:19.855025+0000","last_became_peered":"2023-08-11T15:44:19.855025+0000","last_unstale":"2023-08-11T15:44:25.196608+0000","last_undegraded":"2023-08-11T15:44:25.196608+0000","last_fullsized":"2023-08-11T15:44:25.196608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:52:42.424929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398761+0000","last_change":"2023-08-11T15:44:19.862153+0000","last_active":"2023-08-11T15:44:25.398761+0000","last_peered":"2023-08-11T15:44:25.398761+0000","last_clean":"2023-08-11T15:44:25.398761+0000","last_became_active":"2023-08-11T15:44:19.861830+0000","last_became_peered":"2023-08-11T15:44:19.861830+0000","last_unstale":"2023-08-11T15:44:25.398761+0000","last_undegraded":"2023-08-11T15:44:25.398761+0000","last_fullsized":"2023-08-11T15:44:25.398761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:54:03.374483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240557+0000","last_change":"2023-08-11T15:44:19.857184+0000","last_active":"2023-08-11T15:44:26.240557+0000","last_peered":"2023-08-11T15:44:26.240557+0000","last_clean":"2023-08-11T15:44:26.240557+0000","last_became_active":"2023-08-11T15:44:19.856898+0000","last_became_peered":"2023-08-11T15:44:19.856898+0000","last_unstale":"2023-08-11T15:44:26.240557+0000","last_undegraded":"2023-08-11T15:44:26.240557+0000","last_fullsized":"2023-08-11T15:44:26.240557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:16.949556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240612+0000","last_change":"2023-08-11T15:44:19.852808+0000","last_active":"2023-08-11T15:44:26.240612+0000","last_peered":"2023-08-11T15:44:26.240612+0000","last_clean":"2023-08-11T15:44:26.240612+0000","last_became_active":"2023-08-11T15:44:19.852477+0000","last_became_peered":"2023-08-11T15:44:19.852477+0000","last_unstale":"2023-08-11T15:44:26.240612+0000","last_undegraded":"2023-08-11T15:44:26.240612+0000","last_fullsized":"2023-08-11T15:44:26.240612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:44:30.565373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496877+0000","last_change":"2023-08-11T15:44:19.856273+0000","last_active":"2023-08-11T15:44:27.496877+0000","last_peered":"2023-08-11T15:44:27.496877+0000","last_clean":"2023-08-11T15:44:27.496877+0000","last_became_active":"2023-08-11T15:44:19.854579+0000","last_became_peered":"2023-08-11T15:44:19.854579+0000","last_unstale":"2023-08-11T15:44:27.496877+0000","last_undegraded":"2023-08-11T15:44:27.496877+0000","last_fullsized":"2023-08-11T15:44:27.496877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:49:32.949731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184320+0000","last_change":"2023-08-11T15:44:19.866820+0000","last_active":"2023-08-11T15:44:25.184320+0000","last_peered":"2023-08-11T15:44:25.184320+0000","last_clean":"2023-08-11T15:44:25.184320+0000","last_became_active":"2023-08-11T15:44:19.866507+0000","last_became_peered":"2023-08-11T15:44:19.866507+0000","last_unstale":"2023-08-11T15:44:25.184320+0000","last_undegraded":"2023-08-11T15:44:25.184320+0000","last_fullsized":"2023-08-11T15:44:25.184320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:46.966328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181871+0000","last_change":"2023-08-11T15:44:19.865821+0000","last_active":"2023-08-11T15:44:25.181871+0000","last_peered":"2023-08-11T15:44:25.181871+0000","last_clean":"2023-08-11T15:44:25.181871+0000","last_became_active":"2023-08-11T15:44:19.865510+0000","last_became_peered":"2023-08-11T15:44:19.865510+0000","last_unstale":"2023-08-11T15:44:25.181871+0000","last_undegraded":"2023-08-11T15:44:25.181871+0000","last_fullsized":"2023-08-11T15:44:25.181871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:25:42.382893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130810+0000","last_change":"2023-08-11T15:44:19.863079+0000","last_active":"2023-08-11T15:44:27.130810+0000","last_peered":"2023-08-11T15:44:27.130810+0000","last_clean":"2023-08-11T15:44:27.130810+0000","last_became_active":"2023-08-11T15:44:19.862663+0000","last_became_peered":"2023-08-11T15:44:19.862663+0000","last_unstale":"2023-08-11T15:44:27.130810+0000","last_undegraded":"2023-08-11T15:44:27.130810+0000","last_fullsized":"2023-08-11T15:44:27.130810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:53.635387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130328+0000","last_change":"2023-08-11T15:44:24.201308+0000","last_active":"2023-08-11T15:44:27.130328+0000","last_peered":"2023-08-11T15:44:27.130328+0000","last_clean":"2023-08-11T15:44:27.130328+0000","last_became_active":"2023-08-11T15:44:24.200884+0000","last_became_peered":"2023-08-11T15:44:24.200884+0000","last_unstale":"2023-08-11T15:44:27.130328+0000","last_undegraded":"2023-08-11T15:44:27.130328+0000","last_fullsized":"2023-08-11T15:44:27.130328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:04:43.367558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181892+0000","last_change":"2023-08-11T15:44:19.155621+0000","last_active":"2023-08-11T15:44:25.181892+0000","last_peered":"2023-08-11T15:44:25.181892+0000","last_clean":"2023-08-11T15:44:25.181892+0000","last_became_active":"2023-08-11T15:44:19.154494+0000","last_became_peered":"2023-08-11T15:44:19.154494+0000","last_unstale":"2023-08-11T15:44:25.181892+0000","last_undegraded":"2023-08-11T15:44:25.181892+0000","last_fullsized":"2023-08-11T15:44:25.181892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:22:15.734115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179239+0000","last_change":"2023-08-11T15:44:19.853101+0000","last_active":"2023-08-11T15:44:25.179239+0000","last_peered":"2023-08-11T15:44:25.179239+0000","last_clean":"2023-08-11T15:44:25.179239+0000","last_became_active":"2023-08-11T15:44:19.852804+0000","last_became_peered":"2023-08-11T15:44:19.852804+0000","last_unstale":"2023-08-11T15:44:25.179239+0000","last_undegraded":"2023-08-11T15:44:25.179239+0000","last_fullsized":"2023-08-11T15:44:25.179239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:04:34.124031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196052+0000","last_change":"2023-08-11T15:44:24.198154+0000","last_active":"2023-08-11T15:44:25.196052+0000","last_peered":"2023-08-11T15:44:25.196052+0000","last_clean":"2023-08-11T15:44:25.196052+0000","last_became_active":"2023-08-11T15:44:24.197693+0000","last_became_peered":"2023-08-11T15:44:24.197693+0000","last_unstale":"2023-08-11T15:44:25.196052+0000","last_undegraded":"2023-08-11T15:44:25.196052+0000","last_fullsized":"2023-08-11T15:44:25.196052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:41:28.478746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179250+0000","last_change":"2023-08-11T15:44:19.147219+0000","last_active":"2023-08-11T15:44:25.179250+0000","last_peered":"2023-08-11T15:44:25.179250+0000","last_clean":"2023-08-11T15:44:25.179250+0000","last_became_active":"2023-08-11T15:44:19.146874+0000","last_became_peered":"2023-08-11T15:44:19.146874+0000","last_unstale":"2023-08-11T15:44:25.179250+0000","last_undegraded":"2023-08-11T15:44:25.179250+0000","last_fullsized":"2023-08-11T15:44:25.179250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:22:17.797682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241145+0000","last_change":"2023-08-11T15:44:19.854103+0000","last_active":"2023-08-11T15:44:26.241145+0000","last_peered":"2023-08-11T15:44:26.241145+0000","last_clean":"2023-08-11T15:44:26.241145+0000","last_became_active":"2023-08-11T15:44:19.853542+0000","last_became_peered":"2023-08-11T15:44:19.853542+0000","last_unstale":"2023-08-11T15:44:26.241145+0000","last_undegraded":"2023-08-11T15:44:26.241145+0000","last_fullsized":"2023-08-11T15:44:26.241145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:55:02.787594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196286+0000","last_change":"2023-08-11T15:44:24.202825+0000","last_active":"2023-08-11T15:44:25.196286+0000","last_peered":"2023-08-11T15:44:25.196286+0000","last_clean":"2023-08-11T15:44:25.196286+0000","last_became_active":"2023-08-11T15:44:24.202342+0000","last_became_peered":"2023-08-11T15:44:24.202342+0000","last_unstale":"2023-08-11T15:44:25.196286+0000","last_undegraded":"2023-08-11T15:44:25.196286+0000","last_fullsized":"2023-08-11T15:44:25.196286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:00.909726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196861+0000","last_change":"2023-08-11T15:44:19.148019+0000","last_active":"2023-08-11T15:44:54.196861+0000","last_peered":"2023-08-11T15:44:54.196861+0000","last_clean":"2023-08-11T15:44:54.196861+0000","last_became_active":"2023-08-11T15:44:19.147501+0000","last_became_peered":"2023-08-11T15:44:19.147501+0000","last_unstale":"2023-08-11T15:44:54.196861+0000","last_undegraded":"2023-08-11T15:44:54.196861+0000","last_fullsized":"2023-08-11T15:44:54.196861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:35.232097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497109+0000","last_change":"2023-08-11T15:44:19.865230+0000","last_active":"2023-08-11T15:44:27.497109+0000","last_peered":"2023-08-11T15:44:27.497109+0000","last_clean":"2023-08-11T15:44:27.497109+0000","last_became_active":"2023-08-11T15:44:19.864838+0000","last_became_peered":"2023-08-11T15:44:19.864838+0000","last_unstale":"2023-08-11T15:44:27.497109+0000","last_undegraded":"2023-08-11T15:44:27.497109+0000","last_fullsized":"2023-08-11T15:44:27.497109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:43:54.021675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196125+0000","last_change":"2023-08-11T15:44:24.192282+0000","last_active":"2023-08-11T15:44:25.196125+0000","last_peered":"2023-08-11T15:44:25.196125+0000","last_clean":"2023-08-11T15:44:25.196125+0000","last_became_active":"2023-08-11T15:44:24.191972+0000","last_became_peered":"2023-08-11T15:44:24.191972+0000","last_unstale":"2023-08-11T15:44:25.196125+0000","last_undegraded":"2023-08-11T15:44:25.196125+0000","last_fullsized":"2023-08-11T15:44:25.196125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:39.427988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196635+0000","last_change":"2023-08-11T15:44:19.147181+0000","last_active":"2023-08-11T15:44:54.196635+0000","last_peered":"2023-08-11T15:44:54.196635+0000","last_clean":"2023-08-11T15:44:54.196635+0000","last_became_active":"2023-08-11T15:44:19.146843+0000","last_became_peered":"2023-08-11T15:44:19.146843+0000","last_unstale":"2023-08-11T15:44:54.196635+0000","last_undegraded":"2023-08-11T15:44:54.196635+0000","last_fullsized":"2023-08-11T15:44:54.196635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:36:39.248789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043920+0000","last_change":"2023-08-11T15:44:19.858611+0000","last_active":"2023-08-11T15:44:27.043920+0000","last_peered":"2023-08-11T15:44:27.043920+0000","last_clean":"2023-08-11T15:44:27.043920+0000","last_became_active":"2023-08-11T15:44:19.858288+0000","last_became_peered":"2023-08-11T15:44:19.858288+0000","last_unstale":"2023-08-11T15:44:27.043920+0000","last_undegraded":"2023-08-11T15:44:27.043920+0000","last_fullsized":"2023-08-11T15:44:27.043920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:41.338062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130820+0000","last_change":"2023-08-11T15:44:24.192621+0000","last_active":"2023-08-11T15:44:27.130820+0000","last_peered":"2023-08-11T15:44:27.130820+0000","last_clean":"2023-08-11T15:44:27.130820+0000","last_became_active":"2023-08-11T15:44:24.192183+0000","last_became_peered":"2023-08-11T15:44:24.192183+0000","last_unstale":"2023-08-11T15:44:27.130820+0000","last_undegraded":"2023-08-11T15:44:27.130820+0000","last_fullsized":"2023-08-11T15:44:27.130820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:01.082842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043685+0000","last_change":"2023-08-11T15:44:19.149437+0000","last_active":"2023-08-11T15:44:27.043685+0000","last_peered":"2023-08-11T15:44:27.043685+0000","last_clean":"2023-08-11T15:44:27.043685+0000","last_became_active":"2023-08-11T15:44:19.149086+0000","last_became_peered":"2023-08-11T15:44:19.149086+0000","last_unstale":"2023-08-11T15:44:27.043685+0000","last_undegraded":"2023-08-11T15:44:27.043685+0000","last_fullsized":"2023-08-11T15:44:27.043685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:38.263428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196160+0000","last_change":"2023-08-11T15:44:19.865452+0000","last_active":"2023-08-11T15:44:54.196160+0000","last_peered":"2023-08-11T15:44:54.196160+0000","last_clean":"2023-08-11T15:44:54.196160+0000","last_became_active":"2023-08-11T15:44:19.865133+0000","last_became_peered":"2023-08-11T15:44:19.865133+0000","last_unstale":"2023-08-11T15:44:54.196160+0000","last_undegraded":"2023-08-11T15:44:54.196160+0000","last_fullsized":"2023-08-11T15:44:54.196160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:20.210619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130658+0000","last_change":"2023-08-11T15:44:24.201817+0000","last_active":"2023-08-11T15:44:27.130658+0000","last_peered":"2023-08-11T15:44:27.130658+0000","last_clean":"2023-08-11T15:44:27.130658+0000","last_became_active":"2023-08-11T15:44:24.200766+0000","last_became_peered":"2023-08-11T15:44:24.200766+0000","last_unstale":"2023-08-11T15:44:27.130658+0000","last_undegraded":"2023-08-11T15:44:27.130658+0000","last_fullsized":"2023-08-11T15:44:27.130658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:44:12.538791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909254+0000","last_change":"2023-08-11T15:44:19.151213+0000","last_active":"2023-08-11T15:44:26.909254+0000","last_peered":"2023-08-11T15:44:26.909254+0000","last_clean":"2023-08-11T15:44:26.909254+0000","last_became_active":"2023-08-11T15:44:19.150914+0000","last_became_peered":"2023-08-11T15:44:19.150914+0000","last_unstale":"2023-08-11T15:44:26.909254+0000","last_undegraded":"2023-08-11T15:44:26.909254+0000","last_fullsized":"2023-08-11T15:44:26.909254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:33:38.997413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043814+0000","last_change":"2023-08-11T15:44:19.858238+0000","last_active":"2023-08-11T15:44:27.043814+0000","last_peered":"2023-08-11T15:44:27.043814+0000","last_clean":"2023-08-11T15:44:27.043814+0000","last_became_active":"2023-08-11T15:44:19.857919+0000","last_became_peered":"2023-08-11T15:44:19.857919+0000","last_unstale":"2023-08-11T15:44:27.043814+0000","last_undegraded":"2023-08-11T15:44:27.043814+0000","last_fullsized":"2023-08-11T15:44:27.043814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:20:28.876449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184406+0000","last_change":"2023-08-11T15:44:24.201485+0000","last_active":"2023-08-11T15:44:25.184406+0000","last_peered":"2023-08-11T15:44:25.184406+0000","last_clean":"2023-08-11T15:44:25.184406+0000","last_became_active":"2023-08-11T15:44:24.200984+0000","last_became_peered":"2023-08-11T15:44:24.200984+0000","last_unstale":"2023-08-11T15:44:25.184406+0000","last_undegraded":"2023-08-11T15:44:25.184406+0000","last_fullsized":"2023-08-11T15:44:25.184406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:12.802792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398401+0000","last_change":"2023-08-11T15:44:19.149289+0000","last_active":"2023-08-11T15:44:25.398401+0000","last_peered":"2023-08-11T15:44:25.398401+0000","last_clean":"2023-08-11T15:44:25.398401+0000","last_became_active":"2023-08-11T15:44:19.148881+0000","last_became_peered":"2023-08-11T15:44:19.148881+0000","last_unstale":"2023-08-11T15:44:25.398401+0000","last_undegraded":"2023-08-11T15:44:25.398401+0000","last_fullsized":"2023-08-11T15:44:25.398401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:31.674718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178473+0000","last_change":"2023-08-11T15:44:19.867860+0000","last_active":"2023-08-11T15:44:25.178473+0000","last_peered":"2023-08-11T15:44:25.178473+0000","last_clean":"2023-08-11T15:44:25.178473+0000","last_became_active":"2023-08-11T15:44:19.867545+0000","last_became_peered":"2023-08-11T15:44:19.867545+0000","last_unstale":"2023-08-11T15:44:25.178473+0000","last_undegraded":"2023-08-11T15:44:25.178473+0000","last_fullsized":"2023-08-11T15:44:25.178473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:22.607885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241050+0000","last_change":"2023-08-11T15:44:24.197251+0000","last_active":"2023-08-11T15:44:26.241050+0000","last_peered":"2023-08-11T15:44:26.241050+0000","last_clean":"2023-08-11T15:44:26.241050+0000","last_became_active":"2023-08-11T15:44:24.196735+0000","last_became_peered":"2023-08-11T15:44:24.196735+0000","last_unstale":"2023-08-11T15:44:26.241050+0000","last_undegraded":"2023-08-11T15:44:26.241050+0000","last_fullsized":"2023-08-11T15:44:26.241050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:28:12.008317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043322+0000","last_change":"2023-08-11T15:44:19.146196+0000","last_active":"2023-08-11T15:44:27.043322+0000","last_peered":"2023-08-11T15:44:27.043322+0000","last_clean":"2023-08-11T15:44:27.043322+0000","last_became_active":"2023-08-11T15:44:19.145812+0000","last_became_peered":"2023-08-11T15:44:19.145812+0000","last_unstale":"2023-08-11T15:44:27.043322+0000","last_undegraded":"2023-08-11T15:44:27.043322+0000","last_fullsized":"2023-08-11T15:44:27.043322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:57.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181289+0000","last_change":"2023-08-11T15:44:19.855389+0000","last_active":"2023-08-11T15:44:25.181289+0000","last_peered":"2023-08-11T15:44:25.181289+0000","last_clean":"2023-08-11T15:44:25.181289+0000","last_became_active":"2023-08-11T15:44:19.855063+0000","last_became_peered":"2023-08-11T15:44:19.855063+0000","last_unstale":"2023-08-11T15:44:25.181289+0000","last_undegraded":"2023-08-11T15:44:25.181289+0000","last_fullsized":"2023-08-11T15:44:25.181289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:06:46.738170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178587+0000","last_change":"2023-08-11T15:44:24.192450+0000","last_active":"2023-08-11T15:44:25.178587+0000","last_peered":"2023-08-11T15:44:25.178587+0000","last_clean":"2023-08-11T15:44:25.178587+0000","last_became_active":"2023-08-11T15:44:24.191989+0000","last_became_peered":"2023-08-11T15:44:24.191989+0000","last_unstale":"2023-08-11T15:44:25.178587+0000","last_undegraded":"2023-08-11T15:44:25.178587+0000","last_fullsized":"2023-08-11T15:44:25.178587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:54:21.852954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909733+0000","last_change":"2023-08-11T15:44:19.149002+0000","last_active":"2023-08-11T15:44:26.909733+0000","last_peered":"2023-08-11T15:44:26.909733+0000","last_clean":"2023-08-11T15:44:26.909733+0000","last_became_active":"2023-08-11T15:44:19.148635+0000","last_became_peered":"2023-08-11T15:44:19.148635+0000","last_unstale":"2023-08-11T15:44:26.909733+0000","last_undegraded":"2023-08-11T15:44:26.909733+0000","last_fullsized":"2023-08-11T15:44:26.909733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:02.711671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497221+0000","last_change":"2023-08-11T15:44:19.858067+0000","last_active":"2023-08-11T15:44:27.497221+0000","last_peered":"2023-08-11T15:44:27.497221+0000","last_clean":"2023-08-11T15:44:27.497221+0000","last_became_active":"2023-08-11T15:44:19.856641+0000","last_became_peered":"2023-08-11T15:44:19.856641+0000","last_unstale":"2023-08-11T15:44:27.497221+0000","last_undegraded":"2023-08-11T15:44:27.497221+0000","last_fullsized":"2023-08-11T15:44:27.497221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:29.896919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181349+0000","last_change":"2023-08-11T15:44:24.196806+0000","last_active":"2023-08-11T15:44:25.181349+0000","last_peered":"2023-08-11T15:44:25.181349+0000","last_clean":"2023-08-11T15:44:25.181349+0000","last_became_active":"2023-08-11T15:44:24.195268+0000","last_became_peered":"2023-08-11T15:44:24.195268+0000","last_unstale":"2023-08-11T15:44:25.181349+0000","last_undegraded":"2023-08-11T15:44:25.181349+0000","last_fullsized":"2023-08-11T15:44:25.181349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:45:38.922930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.496972+0000","last_change":"2023-08-11T15:44:54.190469+0000","last_active":"2023-08-11T15:44:54.190469+0000","last_peered":"2023-08-11T15:44:27.496972+0000","last_clean":"2023-08-11T15:44:27.496972+0000","last_became_active":"2023-08-11T15:44:19.144155+0000","last_became_peered":"2023-08-11T15:44:19.144155+0000","last_unstale":"2023-08-11T15:44:27.496972+0000","last_undegraded":"2023-08-11T15:44:27.496972+0000","last_fullsized":"2023-08-11T15:44:27.496972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240742+0000","last_change":"2023-08-11T15:44:19.853881+0000","last_active":"2023-08-11T15:44:26.240742+0000","last_peered":"2023-08-11T15:44:26.240742+0000","last_clean":"2023-08-11T15:44:26.240742+0000","last_became_active":"2023-08-11T15:44:19.853139+0000","last_became_peered":"2023-08-11T15:44:19.853139+0000","last_unstale":"2023-08-11T15:44:26.240742+0000","last_undegraded":"2023-08-11T15:44:26.240742+0000","last_fullsized":"2023-08-11T15:44:26.240742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:05.315621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196399+0000","last_change":"2023-08-11T15:44:24.201620+0000","last_active":"2023-08-11T15:44:25.196399+0000","last_peered":"2023-08-11T15:44:25.196399+0000","last_clean":"2023-08-11T15:44:25.196399+0000","last_became_active":"2023-08-11T15:44:24.201162+0000","last_became_peered":"2023-08-11T15:44:24.201162+0000","last_unstale":"2023-08-11T15:44:25.196399+0000","last_undegraded":"2023-08-11T15:44:25.196399+0000","last_fullsized":"2023-08-11T15:44:25.196399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:54:06.512139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043904+0000","last_change":"2023-08-11T15:44:19.156899+0000","last_active":"2023-08-11T15:44:27.043904+0000","last_peered":"2023-08-11T15:44:27.043904+0000","last_clean":"2023-08-11T15:44:27.043904+0000","last_became_active":"2023-08-11T15:44:19.156643+0000","last_became_peered":"2023-08-11T15:44:19.156643+0000","last_unstale":"2023-08-11T15:44:27.043904+0000","last_undegraded":"2023-08-11T15:44:27.043904+0000","last_fullsized":"2023-08-11T15:44:27.043904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:27.691696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497337+0000","last_change":"2023-08-11T15:44:19.857828+0000","last_active":"2023-08-11T15:44:27.497337+0000","last_peered":"2023-08-11T15:44:27.497337+0000","last_clean":"2023-08-11T15:44:27.497337+0000","last_became_active":"2023-08-11T15:44:19.856781+0000","last_became_peered":"2023-08-11T15:44:19.856781+0000","last_unstale":"2023-08-11T15:44:27.497337+0000","last_undegraded":"2023-08-11T15:44:27.497337+0000","last_fullsized":"2023-08-11T15:44:27.497337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:59.861717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181369+0000","last_change":"2023-08-11T15:44:24.193213+0000","last_active":"2023-08-11T15:44:25.181369+0000","last_peered":"2023-08-11T15:44:25.181369+0000","last_clean":"2023-08-11T15:44:25.181369+0000","last_became_active":"2023-08-11T15:44:24.192819+0000","last_became_peered":"2023-08-11T15:44:24.192819+0000","last_unstale":"2023-08-11T15:44:25.181369+0000","last_undegraded":"2023-08-11T15:44:25.181369+0000","last_fullsized":"2023-08-11T15:44:25.181369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:16.938740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'10","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:52.186980+0000","last_change":"2023-08-11T15:44:19.149753+0000","last_active":"2023-08-11T15:44:52.186980+0000","last_peered":"2023-08-11T15:44:52.186980+0000","last_clean":"2023-08-11T15:44:52.186980+0000","last_became_active":"2023-08-11T15:44:19.149420+0000","last_became_peered":"2023-08-11T15:44:19.149420+0000","last_unstale":"2023-08-11T15:44:52.186980+0000","last_undegraded":"2023-08-11T15:44:52.186980+0000","last_fullsized":"2023-08-11T15:44:52.186980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:58.376378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196006+0000","last_change":"2023-08-11T15:44:19.861342+0000","last_active":"2023-08-11T15:44:54.196006+0000","last_peered":"2023-08-11T15:44:54.196006+0000","last_clean":"2023-08-11T15:44:54.196006+0000","last_became_active":"2023-08-11T15:44:19.860978+0000","last_became_peered":"2023-08-11T15:44:19.860978+0000","last_unstale":"2023-08-11T15:44:54.196006+0000","last_undegraded":"2023-08-11T15:44:54.196006+0000","last_fullsized":"2023-08-11T15:44:54.196006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:52.637866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044274+0000","last_change":"2023-08-11T15:44:24.191504+0000","last_active":"2023-08-11T15:44:27.044274+0000","last_peered":"2023-08-11T15:44:27.044274+0000","last_clean":"2023-08-11T15:44:27.044274+0000","last_became_active":"2023-08-11T15:44:24.191127+0000","last_became_peered":"2023-08-11T15:44:24.191127+0000","last_unstale":"2023-08-11T15:44:27.044274+0000","last_undegraded":"2023-08-11T15:44:27.044274+0000","last_fullsized":"2023-08-11T15:44:27.044274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:11.970278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130897+0000","last_change":"2023-08-11T15:44:19.148354+0000","last_active":"2023-08-11T15:44:27.130897+0000","last_peered":"2023-08-11T15:44:27.130897+0000","last_clean":"2023-08-11T15:44:27.130897+0000","last_became_active":"2023-08-11T15:44:19.148036+0000","last_became_peered":"2023-08-11T15:44:19.148036+0000","last_unstale":"2023-08-11T15:44:27.130897+0000","last_undegraded":"2023-08-11T15:44:27.130897+0000","last_fullsized":"2023-08-11T15:44:27.130897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:33.516998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398521+0000","last_change":"2023-08-11T15:44:19.862409+0000","last_active":"2023-08-11T15:44:25.398521+0000","last_peered":"2023-08-11T15:44:25.398521+0000","last_clean":"2023-08-11T15:44:25.398521+0000","last_became_active":"2023-08-11T15:44:19.861359+0000","last_became_peered":"2023-08-11T15:44:19.861359+0000","last_unstale":"2023-08-11T15:44:25.398521+0000","last_undegraded":"2023-08-11T15:44:25.398521+0000","last_fullsized":"2023-08-11T15:44:25.398521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:41.755533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184424+0000","last_change":"2023-08-11T15:44:24.197634+0000","last_active":"2023-08-11T15:44:25.184424+0000","last_peered":"2023-08-11T15:44:25.184424+0000","last_clean":"2023-08-11T15:44:25.184424+0000","last_became_active":"2023-08-11T15:44:24.197181+0000","last_became_peered":"2023-08-11T15:44:24.197181+0000","last_unstale":"2023-08-11T15:44:25.184424+0000","last_undegraded":"2023-08-11T15:44:25.184424+0000","last_fullsized":"2023-08-11T15:44:25.184424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:23:45.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043675+0000","last_change":"2023-08-11T15:44:19.147550+0000","last_active":"2023-08-11T15:44:27.043675+0000","last_peered":"2023-08-11T15:44:27.043675+0000","last_clean":"2023-08-11T15:44:27.043675+0000","last_became_active":"2023-08-11T15:44:19.147299+0000","last_became_peered":"2023-08-11T15:44:19.147299+0000","last_unstale":"2023-08-11T15:44:27.043675+0000","last_undegraded":"2023-08-11T15:44:27.043675+0000","last_fullsized":"2023-08-11T15:44:27.043675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:56.050406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189968+0000","last_change":"2023-08-11T15:44:19.862664+0000","last_active":"2023-08-11T15:44:54.189968+0000","last_peered":"2023-08-11T15:44:54.189968+0000","last_clean":"2023-08-11T15:44:54.189968+0000","last_became_active":"2023-08-11T15:44:19.862130+0000","last_became_peered":"2023-08-11T15:44:19.862130+0000","last_unstale":"2023-08-11T15:44:54.189968+0000","last_undegraded":"2023-08-11T15:44:54.189968+0000","last_fullsized":"2023-08-11T15:44:54.189968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:02.591318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181786+0000","last_change":"2023-08-11T15:44:24.199851+0000","last_active":"2023-08-11T15:44:25.181786+0000","last_peered":"2023-08-11T15:44:25.181786+0000","last_clean":"2023-08-11T15:44:25.181786+0000","last_became_active":"2023-08-11T15:44:24.199420+0000","last_became_peered":"2023-08-11T15:44:24.199420+0000","last_unstale":"2023-08-11T15:44:25.181786+0000","last_undegraded":"2023-08-11T15:44:25.181786+0000","last_fullsized":"2023-08-11T15:44:25.181786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:36:31.272229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196088+0000","last_change":"2023-08-11T15:44:19.145888+0000","last_active":"2023-08-11T15:44:25.196088+0000","last_peered":"2023-08-11T15:44:25.196088+0000","last_clean":"2023-08-11T15:44:25.196088+0000","last_became_active":"2023-08-11T15:44:19.145459+0000","last_became_peered":"2023-08-11T15:44:19.145459+0000","last_unstale":"2023-08-11T15:44:25.196088+0000","last_undegraded":"2023-08-11T15:44:25.196088+0000","last_fullsized":"2023-08-11T15:44:25.196088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:36.387374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189722+0000","last_change":"2023-08-11T15:44:19.860035+0000","last_active":"2023-08-11T15:44:54.189722+0000","last_peered":"2023-08-11T15:44:54.189722+0000","last_clean":"2023-08-11T15:44:54.189722+0000","last_became_active":"2023-08-11T15:44:19.859696+0000","last_became_peered":"2023-08-11T15:44:19.859696+0000","last_unstale":"2023-08-11T15:44:54.189722+0000","last_undegraded":"2023-08-11T15:44:54.189722+0000","last_fullsized":"2023-08-11T15:44:54.189722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:24.878953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399086+0000","last_change":"2023-08-11T15:44:24.198255+0000","last_active":"2023-08-11T15:44:25.399086+0000","last_peered":"2023-08-11T15:44:25.399086+0000","last_clean":"2023-08-11T15:44:25.399086+0000","last_became_active":"2023-08-11T15:44:24.197825+0000","last_became_peered":"2023-08-11T15:44:24.197825+0000","last_unstale":"2023-08-11T15:44:25.399086+0000","last_undegraded":"2023-08-11T15:44:25.399086+0000","last_fullsized":"2023-08-11T15:44:25.399086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:41:49.976978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496775+0000","last_change":"2023-08-11T15:44:19.151410+0000","last_active":"2023-08-11T15:44:27.496775+0000","last_peered":"2023-08-11T15:44:27.496775+0000","last_clean":"2023-08-11T15:44:27.496775+0000","last_became_active":"2023-08-11T15:44:19.151117+0000","last_became_peered":"2023-08-11T15:44:19.151117+0000","last_unstale":"2023-08-11T15:44:27.496775+0000","last_undegraded":"2023-08-11T15:44:27.496775+0000","last_fullsized":"2023-08-11T15:44:27.496775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:11.127495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043467+0000","last_change":"2023-08-11T15:44:19.853273+0000","last_active":"2023-08-11T15:44:27.043467+0000","last_peered":"2023-08-11T15:44:27.043467+0000","last_clean":"2023-08-11T15:44:27.043467+0000","last_became_active":"2023-08-11T15:44:19.852914+0000","last_became_peered":"2023-08-11T15:44:19.852914+0000","last_unstale":"2023-08-11T15:44:27.043467+0000","last_undegraded":"2023-08-11T15:44:27.043467+0000","last_fullsized":"2023-08-11T15:44:27.043467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:45.110625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043436+0000","last_change":"2023-08-11T15:44:24.194429+0000","last_active":"2023-08-11T15:44:27.043436+0000","last_peered":"2023-08-11T15:44:27.043436+0000","last_clean":"2023-08-11T15:44:27.043436+0000","last_became_active":"2023-08-11T15:44:24.194089+0000","last_became_peered":"2023-08-11T15:44:24.194089+0000","last_unstale":"2023-08-11T15:44:27.043436+0000","last_undegraded":"2023-08-11T15:44:27.043436+0000","last_fullsized":"2023-08-11T15:44:27.043436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:53.530309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194680+0000","last_change":"2023-08-11T15:44:19.151072+0000","last_active":"2023-08-11T15:44:54.194680+0000","last_peered":"2023-08-11T15:44:54.194680+0000","last_clean":"2023-08-11T15:44:54.194680+0000","last_became_active":"2023-08-11T15:44:19.147713+0000","last_became_peered":"2023-08-11T15:44:19.147713+0000","last_unstale":"2023-08-11T15:44:54.194680+0000","last_undegraded":"2023-08-11T15:44:54.194680+0000","last_fullsized":"2023-08-11T15:44:54.194680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:09:42.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184182+0000","last_change":"2023-08-11T15:44:19.862259+0000","last_active":"2023-08-11T15:44:25.184182+0000","last_peered":"2023-08-11T15:44:25.184182+0000","last_clean":"2023-08-11T15:44:25.184182+0000","last_became_active":"2023-08-11T15:44:19.861910+0000","last_became_peered":"2023-08-11T15:44:19.861910+0000","last_unstale":"2023-08-11T15:44:25.184182+0000","last_undegraded":"2023-08-11T15:44:25.184182+0000","last_fullsized":"2023-08-11T15:44:25.184182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:06:41.610675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.497633+0000","last_change":"2023-08-11T15:44:54.195412+0000","last_active":"2023-08-11T15:44:54.195412+0000","last_peered":"2023-08-11T15:44:27.497633+0000","last_clean":"2023-08-11T15:44:27.497633+0000","last_became_active":"2023-08-11T15:44:24.300268+0000","last_became_peered":"2023-08-11T15:44:24.300268+0000","last_unstale":"2023-08-11T15:44:27.497633+0000","last_undegraded":"2023-08-11T15:44:27.497633+0000","last_fullsized":"2023-08-11T15:44:27.497633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181579+0000","last_change":"2023-08-11T15:44:19.152608+0000","last_active":"2023-08-11T15:44:25.181579+0000","last_peered":"2023-08-11T15:44:25.181579+0000","last_clean":"2023-08-11T15:44:25.181579+0000","last_became_active":"2023-08-11T15:44:19.152255+0000","last_became_peered":"2023-08-11T15:44:19.152255+0000","last_unstale":"2023-08-11T15:44:25.181579+0000","last_undegraded":"2023-08-11T15:44:25.181579+0000","last_fullsized":"2023-08-11T15:44:25.181579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:33.299760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909653+0000","last_change":"2023-08-11T15:44:19.864003+0000","last_active":"2023-08-11T15:44:26.909653+0000","last_peered":"2023-08-11T15:44:26.909653+0000","last_clean":"2023-08-11T15:44:26.909653+0000","last_became_active":"2023-08-11T15:44:19.863661+0000","last_became_peered":"2023-08-11T15:44:19.863661+0000","last_unstale":"2023-08-11T15:44:26.909653+0000","last_undegraded":"2023-08-11T15:44:26.909653+0000","last_fullsized":"2023-08-11T15:44:26.909653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:20:35.118164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398869+0000","last_change":"2023-08-11T15:44:24.194123+0000","last_active":"2023-08-11T15:44:25.398869+0000","last_peered":"2023-08-11T15:44:25.398869+0000","last_clean":"2023-08-11T15:44:25.398869+0000","last_became_active":"2023-08-11T15:44:24.193729+0000","last_became_peered":"2023-08-11T15:44:24.193729+0000","last_unstale":"2023-08-11T15:44:25.398869+0000","last_undegraded":"2023-08-11T15:44:25.398869+0000","last_fullsized":"2023-08-11T15:44:25.398869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:11:10.722179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189631+0000","last_change":"2023-08-11T15:44:19.151367+0000","last_active":"2023-08-11T15:44:54.189631+0000","last_peered":"2023-08-11T15:44:54.189631+0000","last_clean":"2023-08-11T15:44:54.189631+0000","last_became_active":"2023-08-11T15:44:19.150991+0000","last_became_peered":"2023-08-11T15:44:19.150991+0000","last_unstale":"2023-08-11T15:44:54.189631+0000","last_undegraded":"2023-08-11T15:44:54.189631+0000","last_fullsized":"2023-08-11T15:44:54.189631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:03:03.249036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909318+0000","last_change":"2023-08-11T15:44:19.858579+0000","last_active":"2023-08-11T15:44:26.909318+0000","last_peered":"2023-08-11T15:44:26.909318+0000","last_clean":"2023-08-11T15:44:26.909318+0000","last_became_active":"2023-08-11T15:44:19.858184+0000","last_became_peered":"2023-08-11T15:44:19.858184+0000","last_unstale":"2023-08-11T15:44:26.909318+0000","last_undegraded":"2023-08-11T15:44:26.909318+0000","last_fullsized":"2023-08-11T15:44:26.909318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:38.034222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399074+0000","last_change":"2023-08-11T15:44:24.300526+0000","last_active":"2023-08-11T15:44:25.399074+0000","last_peered":"2023-08-11T15:44:25.399074+0000","last_clean":"2023-08-11T15:44:25.399074+0000","last_became_active":"2023-08-11T15:44:24.299883+0000","last_became_peered":"2023-08-11T15:44:24.299883+0000","last_unstale":"2023-08-11T15:44:25.399074+0000","last_undegraded":"2023-08-11T15:44:25.399074+0000","last_fullsized":"2023-08-11T15:44:25.399074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:53:45.487528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043468+0000","last_change":"2023-08-11T15:44:19.858116+0000","last_active":"2023-08-11T15:44:27.043468+0000","last_peered":"2023-08-11T15:44:27.043468+0000","last_clean":"2023-08-11T15:44:27.043468+0000","last_became_active":"2023-08-11T15:44:19.857728+0000","last_became_peered":"2023-08-11T15:44:19.857728+0000","last_unstale":"2023-08-11T15:44:27.043468+0000","last_undegraded":"2023-08-11T15:44:27.043468+0000","last_fullsized":"2023-08-11T15:44:27.043468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:56:36.953948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398236+0000","last_change":"2023-08-11T15:44:19.157904+0000","last_active":"2023-08-11T15:44:25.398236+0000","last_peered":"2023-08-11T15:44:25.398236+0000","last_clean":"2023-08-11T15:44:25.398236+0000","last_became_active":"2023-08-11T15:44:19.157497+0000","last_became_peered":"2023-08-11T15:44:19.157497+0000","last_unstale":"2023-08-11T15:44:25.398236+0000","last_undegraded":"2023-08-11T15:44:25.398236+0000","last_fullsized":"2023-08-11T15:44:25.398236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:10.337630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189839+0000","last_change":"2023-08-11T15:44:24.201246+0000","last_active":"2023-08-11T15:44:54.189839+0000","last_peered":"2023-08-11T15:44:54.189839+0000","last_clean":"2023-08-11T15:44:54.189839+0000","last_became_active":"2023-08-11T15:44:24.200810+0000","last_became_peered":"2023-08-11T15:44:24.200810+0000","last_unstale":"2023-08-11T15:44:54.189839+0000","last_undegraded":"2023-08-11T15:44:54.189839+0000","last_fullsized":"2023-08-11T15:44:54.189839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:47:27.192243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178569+0000","last_change":"2023-08-11T15:44:19.846296+0000","last_active":"2023-08-11T15:44:25.178569+0000","last_peered":"2023-08-11T15:44:25.178569+0000","last_clean":"2023-08-11T15:44:25.178569+0000","last_became_active":"2023-08-11T15:44:19.845890+0000","last_became_peered":"2023-08-11T15:44:19.845890+0000","last_unstale":"2023-08-11T15:44:25.178569+0000","last_undegraded":"2023-08-11T15:44:25.178569+0000","last_fullsized":"2023-08-11T15:44:25.178569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:44:22.834705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196213+0000","last_change":"2023-08-11T15:44:19.144202+0000","last_active":"2023-08-11T15:44:25.196213+0000","last_peered":"2023-08-11T15:44:25.196213+0000","last_clean":"2023-08-11T15:44:25.196213+0000","last_became_active":"2023-08-11T15:44:19.143789+0000","last_became_peered":"2023-08-11T15:44:19.143789+0000","last_unstale":"2023-08-11T15:44:25.196213+0000","last_undegraded":"2023-08-11T15:44:25.196213+0000","last_fullsized":"2023-08-11T15:44:25.196213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:57:56.919463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189289+0000","last_change":"2023-08-11T15:43:14.593634+0000","last_active":"2023-08-11T15:44:54.189289+0000","last_peered":"2023-08-11T15:44:54.189289+0000","last_clean":"2023-08-11T15:44:54.189289+0000","last_became_active":"2023-08-11T15:43:14.593164+0000","last_became_peered":"2023-08-11T15:43:14.593164+0000","last_unstale":"2023-08-11T15:44:54.189289+0000","last_undegraded":"2023-08-11T15:44:54.189289+0000","last_fullsized":"2023-08-11T15:44:54.189289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_clean_scrub_stamp":"2023-08-11T15:40:09.009169+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:15.991425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044055+0000","last_change":"2023-08-11T15:44:24.300703+0000","last_active":"2023-08-11T15:44:27.044055+0000","last_peered":"2023-08-11T15:44:27.044055+0000","last_clean":"2023-08-11T15:44:27.044055+0000","last_became_active":"2023-08-11T15:44:24.300309+0000","last_became_peered":"2023-08-11T15:44:24.300309+0000","last_unstale":"2023-08-11T15:44:27.044055+0000","last_undegraded":"2023-08-11T15:44:27.044055+0000","last_fullsized":"2023-08-11T15:44:27.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:41.018379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398223+0000","last_change":"2023-08-11T15:44:19.858934+0000","last_active":"2023-08-11T15:44:25.398223+0000","last_peered":"2023-08-11T15:44:25.398223+0000","last_clean":"2023-08-11T15:44:25.398223+0000","last_became_active":"2023-08-11T15:44:19.858613+0000","last_became_peered":"2023-08-11T15:44:19.858613+0000","last_unstale":"2023-08-11T15:44:25.398223+0000","last_undegraded":"2023-08-11T15:44:25.398223+0000","last_fullsized":"2023-08-11T15:44:25.398223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:07.938228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189544+0000","last_change":"2023-08-11T15:44:19.148113+0000","last_active":"2023-08-11T15:44:54.189544+0000","last_peered":"2023-08-11T15:44:54.189544+0000","last_clean":"2023-08-11T15:44:54.189544+0000","last_became_active":"2023-08-11T15:44:19.147793+0000","last_became_peered":"2023-08-11T15:44:19.147793+0000","last_unstale":"2023-08-11T15:44:54.189544+0000","last_undegraded":"2023-08-11T15:44:54.189544+0000","last_fullsized":"2023-08-11T15:44:54.189544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:07:38.923167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184402+0000","last_change":"2023-08-11T15:44:24.194418+0000","last_active":"2023-08-11T15:44:25.184402+0000","last_peered":"2023-08-11T15:44:25.184402+0000","last_clean":"2023-08-11T15:44:25.184402+0000","last_became_active":"2023-08-11T15:44:24.193943+0000","last_became_peered":"2023-08-11T15:44:24.193943+0000","last_unstale":"2023-08-11T15:44:25.184402+0000","last_undegraded":"2023-08-11T15:44:25.184402+0000","last_fullsized":"2023-08-11T15:44:25.184402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:35.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196145+0000","last_change":"2023-08-11T15:44:19.151476+0000","last_active":"2023-08-11T15:44:54.196145+0000","last_peered":"2023-08-11T15:44:54.196145+0000","last_clean":"2023-08-11T15:44:54.196145+0000","last_became_active":"2023-08-11T15:44:19.151140+0000","last_became_peered":"2023-08-11T15:44:19.151140+0000","last_unstale":"2023-08-11T15:44:54.196145+0000","last_undegraded":"2023-08-11T15:44:54.196145+0000","last_fullsized":"2023-08-11T15:44:54.196145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:08:09.404210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398895+0000","last_change":"2023-08-11T15:44:19.862255+0000","last_active":"2023-08-11T15:44:25.398895+0000","last_peered":"2023-08-11T15:44:25.398895+0000","last_clean":"2023-08-11T15:44:25.398895+0000","last_became_active":"2023-08-11T15:44:19.861907+0000","last_became_peered":"2023-08-11T15:44:19.861907+0000","last_unstale":"2023-08-11T15:44:25.398895+0000","last_undegraded":"2023-08-11T15:44:25.398895+0000","last_fullsized":"2023-08-11T15:44:25.398895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:03.436493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178469+0000","last_change":"2023-08-11T15:44:24.300513+0000","last_active":"2023-08-11T15:44:25.178469+0000","last_peered":"2023-08-11T15:44:25.178469+0000","last_clean":"2023-08-11T15:44:25.178469+0000","last_became_active":"2023-08-11T15:44:24.300074+0000","last_became_peered":"2023-08-11T15:44:24.300074+0000","last_unstale":"2023-08-11T15:44:25.178469+0000","last_undegraded":"2023-08-11T15:44:25.178469+0000","last_fullsized":"2023-08-11T15:44:25.178469+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:16:21.212112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181835+0000","last_change":"2023-08-11T15:44:19.153794+0000","last_active":"2023-08-11T15:44:25.181835+0000","last_peered":"2023-08-11T15:44:25.181835+0000","last_clean":"2023-08-11T15:44:25.181835+0000","last_became_active":"2023-08-11T15:44:19.153392+0000","last_became_peered":"2023-08-11T15:44:19.153392+0000","last_unstale":"2023-08-11T15:44:25.181835+0000","last_undegraded":"2023-08-11T15:44:25.181835+0000","last_fullsized":"2023-08-11T15:44:25.181835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:47.991902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179190+0000","last_change":"2023-08-11T15:44:19.859799+0000","last_active":"2023-08-11T15:44:25.179190+0000","last_peered":"2023-08-11T15:44:25.179190+0000","last_clean":"2023-08-11T15:44:25.179190+0000","last_became_active":"2023-08-11T15:44:19.859411+0000","last_became_peered":"2023-08-11T15:44:19.859411+0000","last_unstale":"2023-08-11T15:44:25.179190+0000","last_undegraded":"2023-08-11T15:44:25.179190+0000","last_fullsized":"2023-08-11T15:44:25.179190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:19.443644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130327+0000","last_change":"2023-08-11T15:44:24.190272+0000","last_active":"2023-08-11T15:44:27.130327+0000","last_peered":"2023-08-11T15:44:27.130327+0000","last_clean":"2023-08-11T15:44:27.130327+0000","last_became_active":"2023-08-11T15:44:24.189849+0000","last_became_peered":"2023-08-11T15:44:24.189849+0000","last_unstale":"2023-08-11T15:44:27.130327+0000","last_undegraded":"2023-08-11T15:44:27.130327+0000","last_fullsized":"2023-08-11T15:44:27.130327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:26:26.603082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196757+0000","last_change":"2023-08-11T15:44:19.154985+0000","last_active":"2023-08-11T15:44:25.196757+0000","last_peered":"2023-08-11T15:44:25.196757+0000","last_clean":"2023-08-11T15:44:25.196757+0000","last_became_active":"2023-08-11T15:44:19.154694+0000","last_became_peered":"2023-08-11T15:44:19.154694+0000","last_unstale":"2023-08-11T15:44:25.196757+0000","last_undegraded":"2023-08-11T15:44:25.196757+0000","last_fullsized":"2023-08-11T15:44:25.196757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:15.737671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241127+0000","last_change":"2023-08-11T15:44:19.852247+0000","last_active":"2023-08-11T15:44:26.241127+0000","last_peered":"2023-08-11T15:44:26.241127+0000","last_clean":"2023-08-11T15:44:26.241127+0000","last_became_active":"2023-08-11T15:44:19.851456+0000","last_became_peered":"2023-08-11T15:44:19.851456+0000","last_unstale":"2023-08-11T15:44:26.241127+0000","last_undegraded":"2023-08-11T15:44:26.241127+0000","last_fullsized":"2023-08-11T15:44:26.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:57:45.705759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196509+0000","last_change":"2023-08-11T15:44:24.300440+0000","last_active":"2023-08-11T15:44:25.196509+0000","last_peered":"2023-08-11T15:44:25.196509+0000","last_clean":"2023-08-11T15:44:25.196509+0000","last_became_active":"2023-08-11T15:44:24.299981+0000","last_became_peered":"2023-08-11T15:44:24.299981+0000","last_unstale":"2023-08-11T15:44:25.196509+0000","last_undegraded":"2023-08-11T15:44:25.196509+0000","last_fullsized":"2023-08-11T15:44:25.196509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:24.190521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496438+0000","last_change":"2023-08-11T15:44:19.151015+0000","last_active":"2023-08-11T15:44:27.496438+0000","last_peered":"2023-08-11T15:44:27.496438+0000","last_clean":"2023-08-11T15:44:27.496438+0000","last_became_active":"2023-08-11T15:44:19.150677+0000","last_became_peered":"2023-08-11T15:44:19.150677+0000","last_unstale":"2023-08-11T15:44:27.496438+0000","last_undegraded":"2023-08-11T15:44:27.496438+0000","last_fullsized":"2023-08-11T15:44:27.496438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:38:34.368253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.195188+0000","last_change":"2023-08-11T15:44:19.861033+0000","last_active":"2023-08-11T15:44:54.195188+0000","last_peered":"2023-08-11T15:44:54.195188+0000","last_clean":"2023-08-11T15:44:54.195188+0000","last_became_active":"2023-08-11T15:44:19.860643+0000","last_became_peered":"2023-08-11T15:44:19.860643+0000","last_unstale":"2023-08-11T15:44:54.195188+0000","last_undegraded":"2023-08-11T15:44:54.195188+0000","last_fullsized":"2023-08-11T15:44:54.195188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:56:51.466526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181930+0000","last_change":"2023-08-11T15:44:24.199389+0000","last_active":"2023-08-11T15:44:25.181930+0000","last_peered":"2023-08-11T15:44:25.181930+0000","last_clean":"2023-08-11T15:44:25.181930+0000","last_became_active":"2023-08-11T15:44:24.196226+0000","last_became_peered":"2023-08-11T15:44:24.196226+0000","last_unstale":"2023-08-11T15:44:25.181930+0000","last_undegraded":"2023-08-11T15:44:25.181930+0000","last_fullsized":"2023-08-11T15:44:25.181930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:03.253768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178850+0000","last_change":"2023-08-11T15:44:19.152895+0000","last_active":"2023-08-11T15:44:25.178850+0000","last_peered":"2023-08-11T15:44:25.178850+0000","last_clean":"2023-08-11T15:44:25.178850+0000","last_became_active":"2023-08-11T15:44:19.152577+0000","last_became_peered":"2023-08-11T15:44:19.152577+0000","last_unstale":"2023-08-11T15:44:25.178850+0000","last_undegraded":"2023-08-11T15:44:25.178850+0000","last_fullsized":"2023-08-11T15:44:25.178850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:41:12.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183940+0000","last_change":"2023-08-11T15:44:19.861101+0000","last_active":"2023-08-11T15:44:25.183940+0000","last_peered":"2023-08-11T15:44:25.183940+0000","last_clean":"2023-08-11T15:44:25.183940+0000","last_became_active":"2023-08-11T15:44:19.860529+0000","last_became_peered":"2023-08-11T15:44:19.860529+0000","last_unstale":"2023-08-11T15:44:25.183940+0000","last_undegraded":"2023-08-11T15:44:25.183940+0000","last_fullsized":"2023-08-11T15:44:25.183940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:25.463560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196480+0000","last_change":"2023-08-11T15:44:24.202856+0000","last_active":"2023-08-11T15:44:25.196480+0000","last_peered":"2023-08-11T15:44:25.196480+0000","last_clean":"2023-08-11T15:44:25.196480+0000","last_became_active":"2023-08-11T15:44:24.202317+0000","last_became_peered":"2023-08-11T15:44:24.202317+0000","last_unstale":"2023-08-11T15:44:25.196480+0000","last_undegraded":"2023-08-11T15:44:25.196480+0000","last_fullsized":"2023-08-11T15:44:25.196480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:12:44.969687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130438+0000","last_change":"2023-08-11T15:44:19.148711+0000","last_active":"2023-08-11T15:44:27.130438+0000","last_peered":"2023-08-11T15:44:27.130438+0000","last_clean":"2023-08-11T15:44:27.130438+0000","last_became_active":"2023-08-11T15:44:19.148361+0000","last_became_peered":"2023-08-11T15:44:19.148361+0000","last_unstale":"2023-08-11T15:44:27.130438+0000","last_undegraded":"2023-08-11T15:44:27.130438+0000","last_fullsized":"2023-08-11T15:44:27.130438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:45.703207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178814+0000","last_change":"2023-08-11T15:44:19.859562+0000","last_active":"2023-08-11T15:44:25.178814+0000","last_peered":"2023-08-11T15:44:25.178814+0000","last_clean":"2023-08-11T15:44:25.178814+0000","last_became_active":"2023-08-11T15:44:19.859243+0000","last_became_peered":"2023-08-11T15:44:19.859243+0000","last_unstale":"2023-08-11T15:44:25.178814+0000","last_undegraded":"2023-08-11T15:44:25.178814+0000","last_fullsized":"2023-08-11T15:44:25.178814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:50.690599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196555+0000","last_change":"2023-08-11T15:44:24.300317+0000","last_active":"2023-08-11T15:44:25.196555+0000","last_peered":"2023-08-11T15:44:25.196555+0000","last_clean":"2023-08-11T15:44:25.196555+0000","last_became_active":"2023-08-11T15:44:24.299841+0000","last_became_peered":"2023-08-11T15:44:24.299841+0000","last_unstale":"2023-08-11T15:44:25.196555+0000","last_undegraded":"2023-08-11T15:44:25.196555+0000","last_fullsized":"2023-08-11T15:44:25.196555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:16.800305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194494+0000","last_change":"2023-08-11T15:44:19.149111+0000","last_active":"2023-08-11T15:44:54.194494+0000","last_peered":"2023-08-11T15:44:54.194494+0000","last_clean":"2023-08-11T15:44:54.194494+0000","last_became_active":"2023-08-11T15:44:19.148173+0000","last_became_peered":"2023-08-11T15:44:19.148173+0000","last_unstale":"2023-08-11T15:44:54.194494+0000","last_undegraded":"2023-08-11T15:44:54.194494+0000","last_fullsized":"2023-08-11T15:44:54.194494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:51.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189482+0000","last_change":"2023-08-11T15:44:19.863724+0000","last_active":"2023-08-11T15:44:54.189482+0000","last_peered":"2023-08-11T15:44:54.189482+0000","last_clean":"2023-08-11T15:44:54.189482+0000","last_became_active":"2023-08-11T15:44:19.863268+0000","last_became_peered":"2023-08-11T15:44:19.863268+0000","last_unstale":"2023-08-11T15:44:54.189482+0000","last_undegraded":"2023-08-11T15:44:54.189482+0000","last_fullsized":"2023-08-11T15:44:54.189482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:27:34.481889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497528+0000","last_change":"2023-08-11T15:44:24.191301+0000","last_active":"2023-08-11T15:44:27.497528+0000","last_peered":"2023-08-11T15:44:27.497528+0000","last_clean":"2023-08-11T15:44:27.497528+0000","last_became_active":"2023-08-11T15:44:24.190686+0000","last_became_peered":"2023-08-11T15:44:24.190686+0000","last_unstale":"2023-08-11T15:44:27.497528+0000","last_undegraded":"2023-08-11T15:44:27.497528+0000","last_fullsized":"2023-08-11T15:44:27.497528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:29:52.896841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130772+0000","last_change":"2023-08-11T15:44:19.147756+0000","last_active":"2023-08-11T15:44:27.130772+0000","last_peered":"2023-08-11T15:44:27.130772+0000","last_clean":"2023-08-11T15:44:27.130772+0000","last_became_active":"2023-08-11T15:44:19.147426+0000","last_became_peered":"2023-08-11T15:44:19.147426+0000","last_unstale":"2023-08-11T15:44:27.130772+0000","last_undegraded":"2023-08-11T15:44:27.130772+0000","last_fullsized":"2023-08-11T15:44:27.130772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:45.863313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190289+0000","last_change":"2023-08-11T15:44:19.863354+0000","last_active":"2023-08-11T15:44:54.190289+0000","last_peered":"2023-08-11T15:44:54.190289+0000","last_clean":"2023-08-11T15:44:54.190289+0000","last_became_active":"2023-08-11T15:44:19.862700+0000","last_became_peered":"2023-08-11T15:44:19.862700+0000","last_unstale":"2023-08-11T15:44:54.190289+0000","last_undegraded":"2023-08-11T15:44:54.190289+0000","last_fullsized":"2023-08-11T15:44:54.190289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:14:44.778310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240828+0000","last_change":"2023-08-11T15:44:24.204657+0000","last_active":"2023-08-11T15:44:26.240828+0000","last_peered":"2023-08-11T15:44:26.240828+0000","last_clean":"2023-08-11T15:44:26.240828+0000","last_became_active":"2023-08-11T15:44:24.204108+0000","last_became_peered":"2023-08-11T15:44:24.204108+0000","last_unstale":"2023-08-11T15:44:26.240828+0000","last_undegraded":"2023-08-11T15:44:26.240828+0000","last_fullsized":"2023-08-11T15:44:26.240828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:17.431253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044145+0000","last_change":"2023-08-11T15:44:19.147014+0000","last_active":"2023-08-11T15:44:27.044145+0000","last_peered":"2023-08-11T15:44:27.044145+0000","last_clean":"2023-08-11T15:44:27.044145+0000","last_became_active":"2023-08-11T15:44:19.146720+0000","last_became_peered":"2023-08-11T15:44:19.146720+0000","last_unstale":"2023-08-11T15:44:27.044145+0000","last_undegraded":"2023-08-11T15:44:27.044145+0000","last_fullsized":"2023-08-11T15:44:27.044145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:42:56.620766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.498380+0000","last_change":"2023-08-11T15:44:54.190492+0000","last_active":"2023-08-11T15:44:54.190492+0000","last_peered":"2023-08-11T15:44:27.498380+0000","last_clean":"2023-08-11T15:44:27.498380+0000","last_became_active":"2023-08-11T15:44:19.853078+0000","last_became_peered":"2023-08-11T15:44:19.853078+0000","last_unstale":"2023-08-11T15:44:27.498380+0000","last_undegraded":"2023-08-11T15:44:27.498380+0000","last_fullsized":"2023-08-11T15:44:27.498380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043195+0000","last_change":"2023-08-11T15:44:24.192709+0000","last_active":"2023-08-11T15:44:27.043195+0000","last_peered":"2023-08-11T15:44:27.043195+0000","last_clean":"2023-08-11T15:44:27.043195+0000","last_became_active":"2023-08-11T15:44:24.192297+0000","last_became_peered":"2023-08-11T15:44:24.192297+0000","last_unstale":"2023-08-11T15:44:27.043195+0000","last_undegraded":"2023-08-11T15:44:27.043195+0000","last_fullsized":"2023-08-11T15:44:27.043195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:39.723615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399018+0000","last_change":"2023-08-11T15:44:19.158301+0000","last_active":"2023-08-11T15:44:25.399018+0000","last_peered":"2023-08-11T15:44:25.399018+0000","last_clean":"2023-08-11T15:44:25.399018+0000","last_became_active":"2023-08-11T15:44:19.157990+0000","last_became_peered":"2023-08-11T15:44:19.157990+0000","last_unstale":"2023-08-11T15:44:25.399018+0000","last_undegraded":"2023-08-11T15:44:25.399018+0000","last_fullsized":"2023-08-11T15:44:25.399018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:02.696501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189944+0000","last_change":"2023-08-11T15:44:19.857357+0000","last_active":"2023-08-11T15:44:54.189944+0000","last_peered":"2023-08-11T15:44:54.189944+0000","last_clean":"2023-08-11T15:44:54.189944+0000","last_became_active":"2023-08-11T15:44:19.857020+0000","last_became_peered":"2023-08-11T15:44:19.857020+0000","last_unstale":"2023-08-11T15:44:54.189944+0000","last_undegraded":"2023-08-11T15:44:54.189944+0000","last_fullsized":"2023-08-11T15:44:54.189944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:37.243586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183978+0000","last_change":"2023-08-11T15:44:19.863314+0000","last_active":"2023-08-11T15:44:25.183978+0000","last_peered":"2023-08-11T15:44:25.183978+0000","last_clean":"2023-08-11T15:44:25.183978+0000","last_became_active":"2023-08-11T15:44:19.862964+0000","last_became_peered":"2023-08-11T15:44:19.862964+0000","last_unstale":"2023-08-11T15:44:25.183978+0000","last_undegraded":"2023-08-11T15:44:25.183978+0000","last_fullsized":"2023-08-11T15:44:25.183978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:16.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130670+0000","last_change":"2023-08-11T15:44:19.850575+0000","last_active":"2023-08-11T15:44:27.130670+0000","last_peered":"2023-08-11T15:44:27.130670+0000","last_clean":"2023-08-11T15:44:27.130670+0000","last_became_active":"2023-08-11T15:44:19.850231+0000","last_became_peered":"2023-08-11T15:44:19.850231+0000","last_unstale":"2023-08-11T15:44:27.130670+0000","last_undegraded":"2023-08-11T15:44:27.130670+0000","last_fullsized":"2023-08-11T15:44:27.130670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:21.005350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130303+0000","last_change":"2023-08-11T15:44:19.857387+0000","last_active":"2023-08-11T15:44:27.130303+0000","last_peered":"2023-08-11T15:44:27.130303+0000","last_clean":"2023-08-11T15:44:27.130303+0000","last_became_active":"2023-08-11T15:44:19.856830+0000","last_became_peered":"2023-08-11T15:44:19.856830+0000","last_unstale":"2023-08-11T15:44:27.130303+0000","last_undegraded":"2023-08-11T15:44:27.130303+0000","last_fullsized":"2023-08-11T15:44:27.130303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:42.512448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241040+0000","last_change":"2023-08-11T15:44:19.868112+0000","last_active":"2023-08-11T15:44:26.241040+0000","last_peered":"2023-08-11T15:44:26.241040+0000","last_clean":"2023-08-11T15:44:26.241040+0000","last_became_active":"2023-08-11T15:44:19.867610+0000","last_became_peered":"2023-08-11T15:44:19.867610+0000","last_unstale":"2023-08-11T15:44:26.241040+0000","last_undegraded":"2023-08-11T15:44:26.241040+0000","last_fullsized":"2023-08-11T15:44:26.241040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:18.810119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398997+0000","last_change":"2023-08-11T15:44:19.858349+0000","last_active":"2023-08-11T15:44:25.398997+0000","last_peered":"2023-08-11T15:44:25.398997+0000","last_clean":"2023-08-11T15:44:25.398997+0000","last_became_active":"2023-08-11T15:44:19.858041+0000","last_became_peered":"2023-08-11T15:44:19.858041+0000","last_unstale":"2023-08-11T15:44:25.398997+0000","last_undegraded":"2023-08-11T15:44:25.398997+0000","last_fullsized":"2023-08-11T15:44:25.398997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:04.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184464+0000","last_change":"2023-08-11T15:44:19.865383+0000","last_active":"2023-08-11T15:44:25.184464+0000","last_peered":"2023-08-11T15:44:25.184464+0000","last_clean":"2023-08-11T15:44:25.184464+0000","last_became_active":"2023-08-11T15:44:19.865065+0000","last_became_peered":"2023-08-11T15:44:19.865065+0000","last_unstale":"2023-08-11T15:44:25.184464+0000","last_undegraded":"2023-08-11T15:44:25.184464+0000","last_fullsized":"2023-08-11T15:44:25.184464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:39:27.425778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130961+0000","last_change":"2023-08-11T15:44:19.859297+0000","last_active":"2023-08-11T15:44:27.130961+0000","last_peered":"2023-08-11T15:44:27.130961+0000","last_clean":"2023-08-11T15:44:27.130961+0000","last_became_active":"2023-08-11T15:44:19.857517+0000","last_became_peered":"2023-08-11T15:44:19.857517+0000","last_unstale":"2023-08-11T15:44:27.130961+0000","last_undegraded":"2023-08-11T15:44:27.130961+0000","last_fullsized":"2023-08-11T15:44:27.130961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:52:53.104396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179277+0000","last_change":"2023-08-11T15:44:19.850485+0000","last_active":"2023-08-11T15:44:25.179277+0000","last_peered":"2023-08-11T15:44:25.179277+0000","last_clean":"2023-08-11T15:44:25.179277+0000","last_became_active":"2023-08-11T15:44:19.850047+0000","last_became_peered":"2023-08-11T15:44:19.850047+0000","last_unstale":"2023-08-11T15:44:25.179277+0000","last_undegraded":"2023-08-11T15:44:25.179277+0000","last_fullsized":"2023-08-11T15:44:25.179277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:11.380061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043361+0000","last_change":"2023-08-11T15:44:19.861953+0000","last_active":"2023-08-11T15:44:27.043361+0000","last_peered":"2023-08-11T15:44:27.043361+0000","last_clean":"2023-08-11T15:44:27.043361+0000","last_became_active":"2023-08-11T15:44:19.861523+0000","last_became_peered":"2023-08-11T15:44:19.861523+0000","last_unstale":"2023-08-11T15:44:27.043361+0000","last_undegraded":"2023-08-11T15:44:27.043361+0000","last_fullsized":"2023-08-11T15:44:27.043361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:14.229621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178755+0000","last_change":"2023-08-11T15:44:19.849283+0000","last_active":"2023-08-11T15:44:25.178755+0000","last_peered":"2023-08-11T15:44:25.178755+0000","last_clean":"2023-08-11T15:44:25.178755+0000","last_became_active":"2023-08-11T15:44:19.848304+0000","last_became_peered":"2023-08-11T15:44:19.848304+0000","last_unstale":"2023-08-11T15:44:25.178755+0000","last_undegraded":"2023-08-11T15:44:25.178755+0000","last_fullsized":"2023-08-11T15:44:25.178755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:06.449360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497606+0000","last_change":"2023-08-11T15:44:19.853532+0000","last_active":"2023-08-11T15:44:27.497606+0000","last_peered":"2023-08-11T15:44:27.497606+0000","last_clean":"2023-08-11T15:44:27.497606+0000","last_became_active":"2023-08-11T15:44:19.853182+0000","last_became_peered":"2023-08-11T15:44:19.853182+0000","last_unstale":"2023-08-11T15:44:27.497606+0000","last_undegraded":"2023-08-11T15:44:27.497606+0000","last_fullsized":"2023-08-11T15:44:27.497606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:51.245443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398519+0000","last_change":"2023-08-11T15:44:19.860571+0000","last_active":"2023-08-11T15:44:25.398519+0000","last_peered":"2023-08-11T15:44:25.398519+0000","last_clean":"2023-08-11T15:44:25.398519+0000","last_became_active":"2023-08-11T15:44:19.858570+0000","last_became_peered":"2023-08-11T15:44:19.858570+0000","last_unstale":"2023-08-11T15:44:25.398519+0000","last_undegraded":"2023-08-11T15:44:25.398519+0000","last_fullsized":"2023-08-11T15:44:25.398519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:37.343811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184137+0000","last_change":"2023-08-11T15:44:19.865988+0000","last_active":"2023-08-11T15:44:25.184137+0000","last_peered":"2023-08-11T15:44:25.184137+0000","last_clean":"2023-08-11T15:44:25.184137+0000","last_became_active":"2023-08-11T15:44:19.865613+0000","last_became_peered":"2023-08-11T15:44:19.865613+0000","last_unstale":"2023-08-11T15:44:25.184137+0000","last_undegraded":"2023-08-11T15:44:25.184137+0000","last_fullsized":"2023-08-11T15:44:25.184137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:07.472727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":48682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.77400000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1061196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.627}]},{"osd":3,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.503}]},{"osd":4,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38400000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.497}]},{"osd":8,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.61299999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83399999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.627},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.751}]},{"osd":6,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.67100000000000004}]},{"osd":8,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.80300000000000005}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":481002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.75800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72499999999999998,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.85799999999999998,"5min":1.05,"15min":1.05},"last":1.1160000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.435,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99199999999999999,"5min":1.077,"15min":1.077},"last":1.145},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0980000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.96899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95399999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":1.1859999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.5819999999999999,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":1.044}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.1299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1075775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.45500000000000002,"15min":0.44800000000000001},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.628},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Aug 11 15:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.79300000000000004,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.498},"max":{"1min":4.8890000000000002,"5min":4.8890000000000002,"15min":4.8890000000000002},"last":0.88}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":2.077,"15min":2.077},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.008,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.581,"15min":1.581},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92500000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.7,"15min":1.7},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0269999999999999,"5min":1.22,"15min":1.22},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1319999999999999,"5min":1.623,"15min":1.623},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.61799999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.63,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.90000000000000002,"15min":0.89200000000000002},"min":{"1min":0.64500000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.85499999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.98399999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.95199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":470965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.76600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88600000000000001,"5min":4.9020000000000001,"15min":4.9020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.45900000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.62}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80800000000000005,"5min":1.593,"15min":1.593},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 15:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.024,"15min":1.024},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74099999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":1.05,"15min":1.05},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.90100000000000002}]},{"osd":8,"last update":"Fri Aug 11 15:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67700000000000005,"15min":0.66600000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1180000000000001,"5min":1.298,"15min":1.298},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.5409999999999999,"5min":6.0609999999999999,"15min":6.0609999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.087,"5min":6.101,"15min":6.101},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.621,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.441,"15min":0.433},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46100000000000002,"15min":0.44700000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.65300000000000002,"15min":0.61299999999999999},"min":{"1min":0.56000000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.54300000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.056,"5min":1.175,"15min":1.175},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.032,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.74399999999999999,"15min":0.72799999999999998},"min":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.78900000000000003,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.85799999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1779999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.149,"5min":1.276,"15min":1.276},"last":1.0489999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.51400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.81999999999999995}]},{"osd":11,"last update":"Fri Aug 11 15:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.016}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":470999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54600000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80300000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.44},"min":{"1min":0.29699999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69499999999999995}]},{"osd":8,"last update":"Fri Aug 11 15:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.879},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.97499999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.95999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":467227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.47699999999999998,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68100000000000005,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.746}]},{"osd":2,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55600000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94399999999999995,"5min":1.105,"15min":1.105},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.53800000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.72399999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.58799999999999997,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Aug 11 15:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.629},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.73599999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95099999999999996,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.266,"5min":5.266,"15min":5.266},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.51000000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.84799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.317},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.35999999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83599999999999997,"5min":1.552,"15min":1.552},"last":0.626},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":1.502,"15min":1.502},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47499999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.016,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93899999999999995,"5min":1.526,"15min":1.526},"last":0.68999999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1057342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79500000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60599999999999998,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83299999999999996,"5min":5.359,"15min":5.359},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.434,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.872,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.443},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77000000000000002,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.36899999999999999},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41999999999999998,"15min":0.40400000000000003},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.73099999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.032,"15min":1.032},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82599999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]},{"osd":8,"last update":"Fri Aug 11 15:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.71599999999999997}]},{"osd":10,"last update":"Fri Aug 11 15:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":21257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:44:55.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:44:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 -- 172.21.15.32:0/1664834463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4073380 msgr2=0x7fe9a4073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/1664834463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4073380 0x7fe9a4073800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe998009990 tx=0x7fe998009ca0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 -- 172.21.15.32:0/1664834463 shutdown_connections 2023-08-11T15:44:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/1664834463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a41087e0 0x7fe9a410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/1664834463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a4074960 0x7fe9a4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/1664834463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4073380 0x7fe9a4073800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 -- 172.21.15.32:0/1664834463 >> 172.21.15.32:0/1664834463 conn(0x7fe9a40fbe50 msgr2=0x7fe9a40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.945+0000 7fe9a91ff700 1 -- 172.21.15.32:0/1664834463 shutdown_connections 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.946+0000 7fe9a91ff700 1 -- 172.21.15.32:0/1664834463 wait complete. 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.946+0000 7fe9a91ff700 1 Processor -- start 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a91ff700 1 -- start start 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a91ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a4073380 0x7fe9a4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a91ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4074960 0x7fe9a4198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a91ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41087e0 0x7fe9a419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a91ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a41a03b0 con 0x7fe9a4074960 2023-08-11T15:44:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a91ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a41a04f0 con 0x7fe9a41087e0 2023-08-11T15:44:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a91ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a41a0680 con 0x7fe9a4073380 2023-08-11T15:44:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4074960 0x7fe9a4198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a4073380 0x7fe9a4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a4073380 0x7fe9a4198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:40576/0 (socket says 172.21.15.32:40576) 2023-08-11T15:44:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a359e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41087e0 0x7fe9a419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.947+0000 7fe9a2d9d700 1 -- 172.21.15.32:0/2754959946 learned_addr learned my addr 172.21.15.32:0/2754959946 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a2d9d700 1 -- 172.21.15.32:0/2754959946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41087e0 msgr2=0x7fe9a419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a2d9d700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41087e0 0x7fe9a419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a2d9d700 1 -- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4074960 msgr2=0x7fe9a4198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a2d9d700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4074960 0x7fe9a4198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a2d9d700 1 -- 172.21.15.32:0/2754959946 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994009520 con 0x7fe9a4073380 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a259c700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4074960 0x7fe9a4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a2d9d700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a4073380 0x7fe9a4198830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe99400eb30 tx=0x7fe99400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe98bfff700 1 -- 172.21.15.32:0/2754959946 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994009880 con 0x7fe9a4073380 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9980095f0 con 0x7fe9a4073380 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe98bfff700 1 -- 172.21.15.32:0/2754959946 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9940044e0 con 0x7fe9a4073380 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe98bfff700 1 -- 172.21.15.32:0/2754959946 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994003d80 con 0x7fe9a4073380 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.948+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a41a0d30 con 0x7fe9a4073380 2023-08-11T15:44:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.949+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a4106e90 con 0x7fe9a4073380 2023-08-11T15:44:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.951+0000 7fe98bfff700 1 -- 172.21.15.32:0/2754959946 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fe994003f00 con 0x7fe9a4073380 2023-08-11T15:44:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.953+0000 7fe98bfff700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe98c0a0020 0x7fe98c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.953+0000 7fe98bfff700 1 -- 172.21.15.32:0/2754959946 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe994014070 con 0x7fe9a4073380 2023-08-11T15:44:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.953+0000 7fe9a259c700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe98c0a0020 0x7fe98c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.954+0000 7fe9a259c700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe98c0a0020 0x7fe98c0a24e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b2e0 tx=0x7fe998005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:55.954+0000 7fe98bfff700 1 -- 172.21.15.32:0/2754959946 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe9940956c0 con 0x7fe9a4073380 2023-08-11T15:44:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.163+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9a402cd60 con 0x7fe98c0a0020 2023-08-11T15:44:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.175+0000 7fe98bfff700 1 -- 172.21.15.32:0/2754959946 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422261 (secure 0 0 0) 0x7fe9a402cd60 con 0x7fe98c0a0020 2023-08-11T15:44:56.182 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe98c0a0020 msgr2=0x7fe98c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe98c0a0020 0x7fe98c0a24e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b2e0 tx=0x7fe998005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a4073380 msgr2=0x7fe9a4198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a4073380 0x7fe9a4198830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe99400eb30 tx=0x7fe99400ee40 comp rx=0 tx=0).stop 2023-08-11T15:44:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 shutdown_connections 2023-08-11T15:44:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe98c0a0020 0x7fe98c0a24e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41087e0 0x7fe9a419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a4073380 0x7fe9a4198830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 --2- 172.21.15.32:0/2754959946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a4074960 0x7fe9a4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 >> 172.21.15.32:0/2754959946 conn(0x7fe9a40fbe50 msgr2=0x7fe9a40fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.180+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 shutdown_connections 2023-08-11T15:44:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.181+0000 7fe9a91ff700 1 -- 172.21.15.32:0/2754959946 wait complete. 2023-08-11T15:44:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-08-11T15:44:56.205 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-08-11T15:44:56.075793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595131,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":147,"num_read_kb":131,"num_write":231,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337400,"kb_used_data":13148,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604424,"statfs":{"total":1151940427776,"available":1151594930176,"internally_reserved":0,"allocated":13463552,"data_stored":7420815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002474"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184488+0000","last_change":"2023-08-11T15:44:19.862927+0000","last_active":"2023-08-11T15:44:25.184488+0000","last_peered":"2023-08-11T15:44:25.184488+0000","last_clean":"2023-08-11T15:44:25.184488+0000","last_became_active":"2023-08-11T15:44:19.862626+0000","last_became_peered":"2023-08-11T15:44:19.862626+0000","last_unstale":"2023-08-11T15:44:25.184488+0000","last_undegraded":"2023-08-11T15:44:25.184488+0000","last_fullsized":"2023-08-11T15:44:25.184488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:58:44.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241036+0000","last_change":"2023-08-11T15:44:19.852455+0000","last_active":"2023-08-11T15:44:26.241036+0000","last_peered":"2023-08-11T15:44:26.241036+0000","last_clean":"2023-08-11T15:44:26.241036+0000","last_became_active":"2023-08-11T15:44:19.851897+0000","last_became_peered":"2023-08-11T15:44:19.851897+0000","last_unstale":"2023-08-11T15:44:26.241036+0000","last_undegraded":"2023-08-11T15:44:26.241036+0000","last_fullsized":"2023-08-11T15:44:26.241036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:20.816842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241013+0000","last_change":"2023-08-11T15:44:19.855232+0000","last_active":"2023-08-11T15:44:26.241013+0000","last_peered":"2023-08-11T15:44:26.241013+0000","last_clean":"2023-08-11T15:44:26.241013+0000","last_became_active":"2023-08-11T15:44:19.854940+0000","last_became_peered":"2023-08-11T15:44:19.854940+0000","last_unstale":"2023-08-11T15:44:26.241013+0000","last_undegraded":"2023-08-11T15:44:26.241013+0000","last_fullsized":"2023-08-11T15:44:26.241013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:42:52.881533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196583+0000","last_change":"2023-08-11T15:44:19.856137+0000","last_active":"2023-08-11T15:44:25.196583+0000","last_peered":"2023-08-11T15:44:25.196583+0000","last_clean":"2023-08-11T15:44:25.196583+0000","last_became_active":"2023-08-11T15:44:19.855852+0000","last_became_peered":"2023-08-11T15:44:19.855852+0000","last_unstale":"2023-08-11T15:44:25.196583+0000","last_undegraded":"2023-08-11T15:44:25.196583+0000","last_fullsized":"2023-08-11T15:44:25.196583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:21.362833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194351+0000","last_change":"2023-08-11T15:44:19.862618+0000","last_active":"2023-08-11T15:44:54.194351+0000","last_peered":"2023-08-11T15:44:54.194351+0000","last_clean":"2023-08-11T15:44:54.194351+0000","last_became_active":"2023-08-11T15:44:19.862265+0000","last_became_peered":"2023-08-11T15:44:19.862265+0000","last_unstale":"2023-08-11T15:44:54.194351+0000","last_undegraded":"2023-08-11T15:44:54.194351+0000","last_fullsized":"2023-08-11T15:44:54.194351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:53.783047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909293+0000","last_change":"2023-08-11T15:44:19.857801+0000","last_active":"2023-08-11T15:44:26.909293+0000","last_peered":"2023-08-11T15:44:26.909293+0000","last_clean":"2023-08-11T15:44:26.909293+0000","last_became_active":"2023-08-11T15:44:19.857388+0000","last_became_peered":"2023-08-11T15:44:19.857388+0000","last_unstale":"2023-08-11T15:44:26.909293+0000","last_undegraded":"2023-08-11T15:44:26.909293+0000","last_fullsized":"2023-08-11T15:44:26.909293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:55:45.696116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189336+0000","last_change":"2023-08-11T15:44:19.859334+0000","last_active":"2023-08-11T15:44:54.189336+0000","last_peered":"2023-08-11T15:44:54.189336+0000","last_clean":"2023-08-11T15:44:54.189336+0000","last_became_active":"2023-08-11T15:44:19.859073+0000","last_became_peered":"2023-08-11T15:44:19.859073+0000","last_unstale":"2023-08-11T15:44:54.189336+0000","last_undegraded":"2023-08-11T15:44:54.189336+0000","last_fullsized":"2023-08-11T15:44:54.189336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:37:20.633727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178676+0000","last_change":"2023-08-11T15:44:19.853064+0000","last_active":"2023-08-11T15:44:25.178676+0000","last_peered":"2023-08-11T15:44:25.178676+0000","last_clean":"2023-08-11T15:44:25.178676+0000","last_became_active":"2023-08-11T15:44:19.852736+0000","last_became_peered":"2023-08-11T15:44:19.852736+0000","last_unstale":"2023-08-11T15:44:25.178676+0000","last_undegraded":"2023-08-11T15:44:25.178676+0000","last_fullsized":"2023-08-11T15:44:25.178676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:31.513054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497471+0000","last_change":"2023-08-11T15:44:19.861912+0000","last_active":"2023-08-11T15:44:27.497471+0000","last_peered":"2023-08-11T15:44:27.497471+0000","last_clean":"2023-08-11T15:44:27.497471+0000","last_became_active":"2023-08-11T15:44:19.861374+0000","last_became_peered":"2023-08-11T15:44:19.861374+0000","last_unstale":"2023-08-11T15:44:27.497471+0000","last_undegraded":"2023-08-11T15:44:27.497471+0000","last_fullsized":"2023-08-11T15:44:27.497471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:55.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909786+0000","last_change":"2023-08-11T15:44:19.858862+0000","last_active":"2023-08-11T15:44:26.909786+0000","last_peered":"2023-08-11T15:44:26.909786+0000","last_clean":"2023-08-11T15:44:26.909786+0000","last_became_active":"2023-08-11T15:44:19.858512+0000","last_became_peered":"2023-08-11T15:44:19.858512+0000","last_unstale":"2023-08-11T15:44:26.909786+0000","last_undegraded":"2023-08-11T15:44:26.909786+0000","last_fullsized":"2023-08-11T15:44:26.909786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:53:27.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190057+0000","last_change":"2023-08-11T15:44:19.864063+0000","last_active":"2023-08-11T15:44:54.190057+0000","last_peered":"2023-08-11T15:44:54.190057+0000","last_clean":"2023-08-11T15:44:54.190057+0000","last_became_active":"2023-08-11T15:44:19.863081+0000","last_became_peered":"2023-08-11T15:44:19.863081+0000","last_unstale":"2023-08-11T15:44:54.190057+0000","last_undegraded":"2023-08-11T15:44:54.190057+0000","last_fullsized":"2023-08-11T15:44:54.190057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:19:47.689166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196608+0000","last_change":"2023-08-11T15:44:19.855347+0000","last_active":"2023-08-11T15:44:25.196608+0000","last_peered":"2023-08-11T15:44:25.196608+0000","last_clean":"2023-08-11T15:44:25.196608+0000","last_became_active":"2023-08-11T15:44:19.855025+0000","last_became_peered":"2023-08-11T15:44:19.855025+0000","last_unstale":"2023-08-11T15:44:25.196608+0000","last_undegraded":"2023-08-11T15:44:25.196608+0000","last_fullsized":"2023-08-11T15:44:25.196608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:52:42.424929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398761+0000","last_change":"2023-08-11T15:44:19.862153+0000","last_active":"2023-08-11T15:44:25.398761+0000","last_peered":"2023-08-11T15:44:25.398761+0000","last_clean":"2023-08-11T15:44:25.398761+0000","last_became_active":"2023-08-11T15:44:19.861830+0000","last_became_peered":"2023-08-11T15:44:19.861830+0000","last_unstale":"2023-08-11T15:44:25.398761+0000","last_undegraded":"2023-08-11T15:44:25.398761+0000","last_fullsized":"2023-08-11T15:44:25.398761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:54:03.374483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240557+0000","last_change":"2023-08-11T15:44:19.857184+0000","last_active":"2023-08-11T15:44:26.240557+0000","last_peered":"2023-08-11T15:44:26.240557+0000","last_clean":"2023-08-11T15:44:26.240557+0000","last_became_active":"2023-08-11T15:44:19.856898+0000","last_became_peered":"2023-08-11T15:44:19.856898+0000","last_unstale":"2023-08-11T15:44:26.240557+0000","last_undegraded":"2023-08-11T15:44:26.240557+0000","last_fullsized":"2023-08-11T15:44:26.240557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:16.949556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240612+0000","last_change":"2023-08-11T15:44:19.852808+0000","last_active":"2023-08-11T15:44:26.240612+0000","last_peered":"2023-08-11T15:44:26.240612+0000","last_clean":"2023-08-11T15:44:26.240612+0000","last_became_active":"2023-08-11T15:44:19.852477+0000","last_became_peered":"2023-08-11T15:44:19.852477+0000","last_unstale":"2023-08-11T15:44:26.240612+0000","last_undegraded":"2023-08-11T15:44:26.240612+0000","last_fullsized":"2023-08-11T15:44:26.240612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:44:30.565373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496877+0000","last_change":"2023-08-11T15:44:19.856273+0000","last_active":"2023-08-11T15:44:27.496877+0000","last_peered":"2023-08-11T15:44:27.496877+0000","last_clean":"2023-08-11T15:44:27.496877+0000","last_became_active":"2023-08-11T15:44:19.854579+0000","last_became_peered":"2023-08-11T15:44:19.854579+0000","last_unstale":"2023-08-11T15:44:27.496877+0000","last_undegraded":"2023-08-11T15:44:27.496877+0000","last_fullsized":"2023-08-11T15:44:27.496877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:49:32.949731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184320+0000","last_change":"2023-08-11T15:44:19.866820+0000","last_active":"2023-08-11T15:44:25.184320+0000","last_peered":"2023-08-11T15:44:25.184320+0000","last_clean":"2023-08-11T15:44:25.184320+0000","last_became_active":"2023-08-11T15:44:19.866507+0000","last_became_peered":"2023-08-11T15:44:19.866507+0000","last_unstale":"2023-08-11T15:44:25.184320+0000","last_undegraded":"2023-08-11T15:44:25.184320+0000","last_fullsized":"2023-08-11T15:44:25.184320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:46.966328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181871+0000","last_change":"2023-08-11T15:44:19.865821+0000","last_active":"2023-08-11T15:44:25.181871+0000","last_peered":"2023-08-11T15:44:25.181871+0000","last_clean":"2023-08-11T15:44:25.181871+0000","last_became_active":"2023-08-11T15:44:19.865510+0000","last_became_peered":"2023-08-11T15:44:19.865510+0000","last_unstale":"2023-08-11T15:44:25.181871+0000","last_undegraded":"2023-08-11T15:44:25.181871+0000","last_fullsized":"2023-08-11T15:44:25.181871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:25:42.382893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130810+0000","last_change":"2023-08-11T15:44:19.863079+0000","last_active":"2023-08-11T15:44:27.130810+0000","last_peered":"2023-08-11T15:44:27.130810+0000","last_clean":"2023-08-11T15:44:27.130810+0000","last_became_active":"2023-08-11T15:44:19.862663+0000","last_became_peered":"2023-08-11T15:44:19.862663+0000","last_unstale":"2023-08-11T15:44:27.130810+0000","last_undegraded":"2023-08-11T15:44:27.130810+0000","last_fullsized":"2023-08-11T15:44:27.130810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:53.635387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130328+0000","last_change":"2023-08-11T15:44:24.201308+0000","last_active":"2023-08-11T15:44:27.130328+0000","last_peered":"2023-08-11T15:44:27.130328+0000","last_clean":"2023-08-11T15:44:27.130328+0000","last_became_active":"2023-08-11T15:44:24.200884+0000","last_became_peered":"2023-08-11T15:44:24.200884+0000","last_unstale":"2023-08-11T15:44:27.130328+0000","last_undegraded":"2023-08-11T15:44:27.130328+0000","last_fullsized":"2023-08-11T15:44:27.130328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:04:43.367558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181892+0000","last_change":"2023-08-11T15:44:19.155621+0000","last_active":"2023-08-11T15:44:25.181892+0000","last_peered":"2023-08-11T15:44:25.181892+0000","last_clean":"2023-08-11T15:44:25.181892+0000","last_became_active":"2023-08-11T15:44:19.154494+0000","last_became_peered":"2023-08-11T15:44:19.154494+0000","last_unstale":"2023-08-11T15:44:25.181892+0000","last_undegraded":"2023-08-11T15:44:25.181892+0000","last_fullsized":"2023-08-11T15:44:25.181892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:22:15.734115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179239+0000","last_change":"2023-08-11T15:44:19.853101+0000","last_active":"2023-08-11T15:44:25.179239+0000","last_peered":"2023-08-11T15:44:25.179239+0000","last_clean":"2023-08-11T15:44:25.179239+0000","last_became_active":"2023-08-11T15:44:19.852804+0000","last_became_peered":"2023-08-11T15:44:19.852804+0000","last_unstale":"2023-08-11T15:44:25.179239+0000","last_undegraded":"2023-08-11T15:44:25.179239+0000","last_fullsized":"2023-08-11T15:44:25.179239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:04:34.124031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196052+0000","last_change":"2023-08-11T15:44:24.198154+0000","last_active":"2023-08-11T15:44:25.196052+0000","last_peered":"2023-08-11T15:44:25.196052+0000","last_clean":"2023-08-11T15:44:25.196052+0000","last_became_active":"2023-08-11T15:44:24.197693+0000","last_became_peered":"2023-08-11T15:44:24.197693+0000","last_unstale":"2023-08-11T15:44:25.196052+0000","last_undegraded":"2023-08-11T15:44:25.196052+0000","last_fullsized":"2023-08-11T15:44:25.196052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:41:28.478746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179250+0000","last_change":"2023-08-11T15:44:19.147219+0000","last_active":"2023-08-11T15:44:25.179250+0000","last_peered":"2023-08-11T15:44:25.179250+0000","last_clean":"2023-08-11T15:44:25.179250+0000","last_became_active":"2023-08-11T15:44:19.146874+0000","last_became_peered":"2023-08-11T15:44:19.146874+0000","last_unstale":"2023-08-11T15:44:25.179250+0000","last_undegraded":"2023-08-11T15:44:25.179250+0000","last_fullsized":"2023-08-11T15:44:25.179250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:22:17.797682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241145+0000","last_change":"2023-08-11T15:44:19.854103+0000","last_active":"2023-08-11T15:44:26.241145+0000","last_peered":"2023-08-11T15:44:26.241145+0000","last_clean":"2023-08-11T15:44:26.241145+0000","last_became_active":"2023-08-11T15:44:19.853542+0000","last_became_peered":"2023-08-11T15:44:19.853542+0000","last_unstale":"2023-08-11T15:44:26.241145+0000","last_undegraded":"2023-08-11T15:44:26.241145+0000","last_fullsized":"2023-08-11T15:44:26.241145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:55:02.787594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196286+0000","last_change":"2023-08-11T15:44:24.202825+0000","last_active":"2023-08-11T15:44:25.196286+0000","last_peered":"2023-08-11T15:44:25.196286+0000","last_clean":"2023-08-11T15:44:25.196286+0000","last_became_active":"2023-08-11T15:44:24.202342+0000","last_became_peered":"2023-08-11T15:44:24.202342+0000","last_unstale":"2023-08-11T15:44:25.196286+0000","last_undegraded":"2023-08-11T15:44:25.196286+0000","last_fullsized":"2023-08-11T15:44:25.196286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:00.909726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196861+0000","last_change":"2023-08-11T15:44:19.148019+0000","last_active":"2023-08-11T15:44:54.196861+0000","last_peered":"2023-08-11T15:44:54.196861+0000","last_clean":"2023-08-11T15:44:54.196861+0000","last_became_active":"2023-08-11T15:44:19.147501+0000","last_became_peered":"2023-08-11T15:44:19.147501+0000","last_unstale":"2023-08-11T15:44:54.196861+0000","last_undegraded":"2023-08-11T15:44:54.196861+0000","last_fullsized":"2023-08-11T15:44:54.196861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:35.232097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497109+0000","last_change":"2023-08-11T15:44:19.865230+0000","last_active":"2023-08-11T15:44:27.497109+0000","last_peered":"2023-08-11T15:44:27.497109+0000","last_clean":"2023-08-11T15:44:27.497109+0000","last_became_active":"2023-08-11T15:44:19.864838+0000","last_became_peered":"2023-08-11T15:44:19.864838+0000","last_unstale":"2023-08-11T15:44:27.497109+0000","last_undegraded":"2023-08-11T15:44:27.497109+0000","last_fullsized":"2023-08-11T15:44:27.497109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:43:54.021675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196125+0000","last_change":"2023-08-11T15:44:24.192282+0000","last_active":"2023-08-11T15:44:25.196125+0000","last_peered":"2023-08-11T15:44:25.196125+0000","last_clean":"2023-08-11T15:44:25.196125+0000","last_became_active":"2023-08-11T15:44:24.191972+0000","last_became_peered":"2023-08-11T15:44:24.191972+0000","last_unstale":"2023-08-11T15:44:25.196125+0000","last_undegraded":"2023-08-11T15:44:25.196125+0000","last_fullsized":"2023-08-11T15:44:25.196125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:39.427988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196635+0000","last_change":"2023-08-11T15:44:19.147181+0000","last_active":"2023-08-11T15:44:54.196635+0000","last_peered":"2023-08-11T15:44:54.196635+0000","last_clean":"2023-08-11T15:44:54.196635+0000","last_became_active":"2023-08-11T15:44:19.146843+0000","last_became_peered":"2023-08-11T15:44:19.146843+0000","last_unstale":"2023-08-11T15:44:54.196635+0000","last_undegraded":"2023-08-11T15:44:54.196635+0000","last_fullsized":"2023-08-11T15:44:54.196635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:36:39.248789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043920+0000","last_change":"2023-08-11T15:44:19.858611+0000","last_active":"2023-08-11T15:44:27.043920+0000","last_peered":"2023-08-11T15:44:27.043920+0000","last_clean":"2023-08-11T15:44:27.043920+0000","last_became_active":"2023-08-11T15:44:19.858288+0000","last_became_peered":"2023-08-11T15:44:19.858288+0000","last_unstale":"2023-08-11T15:44:27.043920+0000","last_undegraded":"2023-08-11T15:44:27.043920+0000","last_fullsized":"2023-08-11T15:44:27.043920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:41.338062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130820+0000","last_change":"2023-08-11T15:44:24.192621+0000","last_active":"2023-08-11T15:44:27.130820+0000","last_peered":"2023-08-11T15:44:27.130820+0000","last_clean":"2023-08-11T15:44:27.130820+0000","last_became_active":"2023-08-11T15:44:24.192183+0000","last_became_peered":"2023-08-11T15:44:24.192183+0000","last_unstale":"2023-08-11T15:44:27.130820+0000","last_undegraded":"2023-08-11T15:44:27.130820+0000","last_fullsized":"2023-08-11T15:44:27.130820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:01.082842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043685+0000","last_change":"2023-08-11T15:44:19.149437+0000","last_active":"2023-08-11T15:44:27.043685+0000","last_peered":"2023-08-11T15:44:27.043685+0000","last_clean":"2023-08-11T15:44:27.043685+0000","last_became_active":"2023-08-11T15:44:19.149086+0000","last_became_peered":"2023-08-11T15:44:19.149086+0000","last_unstale":"2023-08-11T15:44:27.043685+0000","last_undegraded":"2023-08-11T15:44:27.043685+0000","last_fullsized":"2023-08-11T15:44:27.043685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:38.263428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196160+0000","last_change":"2023-08-11T15:44:19.865452+0000","last_active":"2023-08-11T15:44:54.196160+0000","last_peered":"2023-08-11T15:44:54.196160+0000","last_clean":"2023-08-11T15:44:54.196160+0000","last_became_active":"2023-08-11T15:44:19.865133+0000","last_became_peered":"2023-08-11T15:44:19.865133+0000","last_unstale":"2023-08-11T15:44:54.196160+0000","last_undegraded":"2023-08-11T15:44:54.196160+0000","last_fullsized":"2023-08-11T15:44:54.196160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:20.210619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130658+0000","last_change":"2023-08-11T15:44:24.201817+0000","last_active":"2023-08-11T15:44:27.130658+0000","last_peered":"2023-08-11T15:44:27.130658+0000","last_clean":"2023-08-11T15:44:27.130658+0000","last_became_active":"2023-08-11T15:44:24.200766+0000","last_became_peered":"2023-08-11T15:44:24.200766+0000","last_unstale":"2023-08-11T15:44:27.130658+0000","last_undegraded":"2023-08-11T15:44:27.130658+0000","last_fullsized":"2023-08-11T15:44:27.130658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:44:12.538791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909254+0000","last_change":"2023-08-11T15:44:19.151213+0000","last_active":"2023-08-11T15:44:26.909254+0000","last_peered":"2023-08-11T15:44:26.909254+0000","last_clean":"2023-08-11T15:44:26.909254+0000","last_became_active":"2023-08-11T15:44:19.150914+0000","last_became_peered":"2023-08-11T15:44:19.150914+0000","last_unstale":"2023-08-11T15:44:26.909254+0000","last_undegraded":"2023-08-11T15:44:26.909254+0000","last_fullsized":"2023-08-11T15:44:26.909254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:33:38.997413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043814+0000","last_change":"2023-08-11T15:44:19.858238+0000","last_active":"2023-08-11T15:44:27.043814+0000","last_peered":"2023-08-11T15:44:27.043814+0000","last_clean":"2023-08-11T15:44:27.043814+0000","last_became_active":"2023-08-11T15:44:19.857919+0000","last_became_peered":"2023-08-11T15:44:19.857919+0000","last_unstale":"2023-08-11T15:44:27.043814+0000","last_undegraded":"2023-08-11T15:44:27.043814+0000","last_fullsized":"2023-08-11T15:44:27.043814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:20:28.876449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184406+0000","last_change":"2023-08-11T15:44:24.201485+0000","last_active":"2023-08-11T15:44:25.184406+0000","last_peered":"2023-08-11T15:44:25.184406+0000","last_clean":"2023-08-11T15:44:25.184406+0000","last_became_active":"2023-08-11T15:44:24.200984+0000","last_became_peered":"2023-08-11T15:44:24.200984+0000","last_unstale":"2023-08-11T15:44:25.184406+0000","last_undegraded":"2023-08-11T15:44:25.184406+0000","last_fullsized":"2023-08-11T15:44:25.184406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:12.802792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398401+0000","last_change":"2023-08-11T15:44:19.149289+0000","last_active":"2023-08-11T15:44:25.398401+0000","last_peered":"2023-08-11T15:44:25.398401+0000","last_clean":"2023-08-11T15:44:25.398401+0000","last_became_active":"2023-08-11T15:44:19.148881+0000","last_became_peered":"2023-08-11T15:44:19.148881+0000","last_unstale":"2023-08-11T15:44:25.398401+0000","last_undegraded":"2023-08-11T15:44:25.398401+0000","last_fullsized":"2023-08-11T15:44:25.398401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:31.674718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178473+0000","last_change":"2023-08-11T15:44:19.867860+0000","last_active":"2023-08-11T15:44:25.178473+0000","last_peered":"2023-08-11T15:44:25.178473+0000","last_clean":"2023-08-11T15:44:25.178473+0000","last_became_active":"2023-08-11T15:44:19.867545+0000","last_became_peered":"2023-08-11T15:44:19.867545+0000","last_unstale":"2023-08-11T15:44:25.178473+0000","last_undegraded":"2023-08-11T15:44:25.178473+0000","last_fullsized":"2023-08-11T15:44:25.178473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:22.607885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241050+0000","last_change":"2023-08-11T15:44:24.197251+0000","last_active":"2023-08-11T15:44:26.241050+0000","last_peered":"2023-08-11T15:44:26.241050+0000","last_clean":"2023-08-11T15:44:26.241050+0000","last_became_active":"2023-08-11T15:44:24.196735+0000","last_became_peered":"2023-08-11T15:44:24.196735+0000","last_unstale":"2023-08-11T15:44:26.241050+0000","last_undegraded":"2023-08-11T15:44:26.241050+0000","last_fullsized":"2023-08-11T15:44:26.241050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:28:12.008317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043322+0000","last_change":"2023-08-11T15:44:19.146196+0000","last_active":"2023-08-11T15:44:27.043322+0000","last_peered":"2023-08-11T15:44:27.043322+0000","last_clean":"2023-08-11T15:44:27.043322+0000","last_became_active":"2023-08-11T15:44:19.145812+0000","last_became_peered":"2023-08-11T15:44:19.145812+0000","last_unstale":"2023-08-11T15:44:27.043322+0000","last_undegraded":"2023-08-11T15:44:27.043322+0000","last_fullsized":"2023-08-11T15:44:27.043322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:57.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181289+0000","last_change":"2023-08-11T15:44:19.855389+0000","last_active":"2023-08-11T15:44:25.181289+0000","last_peered":"2023-08-11T15:44:25.181289+0000","last_clean":"2023-08-11T15:44:25.181289+0000","last_became_active":"2023-08-11T15:44:19.855063+0000","last_became_peered":"2023-08-11T15:44:19.855063+0000","last_unstale":"2023-08-11T15:44:25.181289+0000","last_undegraded":"2023-08-11T15:44:25.181289+0000","last_fullsized":"2023-08-11T15:44:25.181289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:06:46.738170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178587+0000","last_change":"2023-08-11T15:44:24.192450+0000","last_active":"2023-08-11T15:44:25.178587+0000","last_peered":"2023-08-11T15:44:25.178587+0000","last_clean":"2023-08-11T15:44:25.178587+0000","last_became_active":"2023-08-11T15:44:24.191989+0000","last_became_peered":"2023-08-11T15:44:24.191989+0000","last_unstale":"2023-08-11T15:44:25.178587+0000","last_undegraded":"2023-08-11T15:44:25.178587+0000","last_fullsized":"2023-08-11T15:44:25.178587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:54:21.852954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909733+0000","last_change":"2023-08-11T15:44:19.149002+0000","last_active":"2023-08-11T15:44:26.909733+0000","last_peered":"2023-08-11T15:44:26.909733+0000","last_clean":"2023-08-11T15:44:26.909733+0000","last_became_active":"2023-08-11T15:44:19.148635+0000","last_became_peered":"2023-08-11T15:44:19.148635+0000","last_unstale":"2023-08-11T15:44:26.909733+0000","last_undegraded":"2023-08-11T15:44:26.909733+0000","last_fullsized":"2023-08-11T15:44:26.909733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:02.711671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497221+0000","last_change":"2023-08-11T15:44:19.858067+0000","last_active":"2023-08-11T15:44:27.497221+0000","last_peered":"2023-08-11T15:44:27.497221+0000","last_clean":"2023-08-11T15:44:27.497221+0000","last_became_active":"2023-08-11T15:44:19.856641+0000","last_became_peered":"2023-08-11T15:44:19.856641+0000","last_unstale":"2023-08-11T15:44:27.497221+0000","last_undegraded":"2023-08-11T15:44:27.497221+0000","last_fullsized":"2023-08-11T15:44:27.497221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:29.896919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181349+0000","last_change":"2023-08-11T15:44:24.196806+0000","last_active":"2023-08-11T15:44:25.181349+0000","last_peered":"2023-08-11T15:44:25.181349+0000","last_clean":"2023-08-11T15:44:25.181349+0000","last_became_active":"2023-08-11T15:44:24.195268+0000","last_became_peered":"2023-08-11T15:44:24.195268+0000","last_unstale":"2023-08-11T15:44:25.181349+0000","last_undegraded":"2023-08-11T15:44:25.181349+0000","last_fullsized":"2023-08-11T15:44:25.181349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:45:38.922930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.496972+0000","last_change":"2023-08-11T15:44:54.190469+0000","last_active":"2023-08-11T15:44:54.190469+0000","last_peered":"2023-08-11T15:44:27.496972+0000","last_clean":"2023-08-11T15:44:27.496972+0000","last_became_active":"2023-08-11T15:44:19.144155+0000","last_became_peered":"2023-08-11T15:44:19.144155+0000","last_unstale":"2023-08-11T15:44:27.496972+0000","last_undegraded":"2023-08-11T15:44:27.496972+0000","last_fullsized":"2023-08-11T15:44:27.496972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240742+0000","last_change":"2023-08-11T15:44:19.853881+0000","last_active":"2023-08-11T15:44:26.240742+0000","last_peered":"2023-08-11T15:44:26.240742+0000","last_clean":"2023-08-11T15:44:26.240742+0000","last_became_active":"2023-08-11T15:44:19.853139+0000","last_became_peered":"2023-08-11T15:44:19.853139+0000","last_unstale":"2023-08-11T15:44:26.240742+0000","last_undegraded":"2023-08-11T15:44:26.240742+0000","last_fullsized":"2023-08-11T15:44:26.240742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:05.315621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196399+0000","last_change":"2023-08-11T15:44:24.201620+0000","last_active":"2023-08-11T15:44:25.196399+0000","last_peered":"2023-08-11T15:44:25.196399+0000","last_clean":"2023-08-11T15:44:25.196399+0000","last_became_active":"2023-08-11T15:44:24.201162+0000","last_became_peered":"2023-08-11T15:44:24.201162+0000","last_unstale":"2023-08-11T15:44:25.196399+0000","last_undegraded":"2023-08-11T15:44:25.196399+0000","last_fullsized":"2023-08-11T15:44:25.196399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:54:06.512139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043904+0000","last_change":"2023-08-11T15:44:19.156899+0000","last_active":"2023-08-11T15:44:27.043904+0000","last_peered":"2023-08-11T15:44:27.043904+0000","last_clean":"2023-08-11T15:44:27.043904+0000","last_became_active":"2023-08-11T15:44:19.156643+0000","last_became_peered":"2023-08-11T15:44:19.156643+0000","last_unstale":"2023-08-11T15:44:27.043904+0000","last_undegraded":"2023-08-11T15:44:27.043904+0000","last_fullsized":"2023-08-11T15:44:27.043904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:27.691696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497337+0000","last_change":"2023-08-11T15:44:19.857828+0000","last_active":"2023-08-11T15:44:27.497337+0000","last_peered":"2023-08-11T15:44:27.497337+0000","last_clean":"2023-08-11T15:44:27.497337+0000","last_became_active":"2023-08-11T15:44:19.856781+0000","last_became_peered":"2023-08-11T15:44:19.856781+0000","last_unstale":"2023-08-11T15:44:27.497337+0000","last_undegraded":"2023-08-11T15:44:27.497337+0000","last_fullsized":"2023-08-11T15:44:27.497337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:59.861717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181369+0000","last_change":"2023-08-11T15:44:24.193213+0000","last_active":"2023-08-11T15:44:25.181369+0000","last_peered":"2023-08-11T15:44:25.181369+0000","last_clean":"2023-08-11T15:44:25.181369+0000","last_became_active":"2023-08-11T15:44:24.192819+0000","last_became_peered":"2023-08-11T15:44:24.192819+0000","last_unstale":"2023-08-11T15:44:25.181369+0000","last_undegraded":"2023-08-11T15:44:25.181369+0000","last_fullsized":"2023-08-11T15:44:25.181369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:16.938740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'10","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:52.186980+0000","last_change":"2023-08-11T15:44:19.149753+0000","last_active":"2023-08-11T15:44:52.186980+0000","last_peered":"2023-08-11T15:44:52.186980+0000","last_clean":"2023-08-11T15:44:52.186980+0000","last_became_active":"2023-08-11T15:44:19.149420+0000","last_became_peered":"2023-08-11T15:44:19.149420+0000","last_unstale":"2023-08-11T15:44:52.186980+0000","last_undegraded":"2023-08-11T15:44:52.186980+0000","last_fullsized":"2023-08-11T15:44:52.186980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:58.376378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196006+0000","last_change":"2023-08-11T15:44:19.861342+0000","last_active":"2023-08-11T15:44:54.196006+0000","last_peered":"2023-08-11T15:44:54.196006+0000","last_clean":"2023-08-11T15:44:54.196006+0000","last_became_active":"2023-08-11T15:44:19.860978+0000","last_became_peered":"2023-08-11T15:44:19.860978+0000","last_unstale":"2023-08-11T15:44:54.196006+0000","last_undegraded":"2023-08-11T15:44:54.196006+0000","last_fullsized":"2023-08-11T15:44:54.196006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:52.637866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044274+0000","last_change":"2023-08-11T15:44:24.191504+0000","last_active":"2023-08-11T15:44:27.044274+0000","last_peered":"2023-08-11T15:44:27.044274+0000","last_clean":"2023-08-11T15:44:27.044274+0000","last_became_active":"2023-08-11T15:44:24.191127+0000","last_became_peered":"2023-08-11T15:44:24.191127+0000","last_unstale":"2023-08-11T15:44:27.044274+0000","last_undegraded":"2023-08-11T15:44:27.044274+0000","last_fullsized":"2023-08-11T15:44:27.044274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:11.970278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130897+0000","last_change":"2023-08-11T15:44:19.148354+0000","last_active":"2023-08-11T15:44:27.130897+0000","last_peered":"2023-08-11T15:44:27.130897+0000","last_clean":"2023-08-11T15:44:27.130897+0000","last_became_active":"2023-08-11T15:44:19.148036+0000","last_became_peered":"2023-08-11T15:44:19.148036+0000","last_unstale":"2023-08-11T15:44:27.130897+0000","last_undegraded":"2023-08-11T15:44:27.130897+0000","last_fullsized":"2023-08-11T15:44:27.130897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:33.516998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398521+0000","last_change":"2023-08-11T15:44:19.862409+0000","last_active":"2023-08-11T15:44:25.398521+0000","last_peered":"2023-08-11T15:44:25.398521+0000","last_clean":"2023-08-11T15:44:25.398521+0000","last_became_active":"2023-08-11T15:44:19.861359+0000","last_became_peered":"2023-08-11T15:44:19.861359+0000","last_unstale":"2023-08-11T15:44:25.398521+0000","last_undegraded":"2023-08-11T15:44:25.398521+0000","last_fullsized":"2023-08-11T15:44:25.398521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:41.755533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184424+0000","last_change":"2023-08-11T15:44:24.197634+0000","last_active":"2023-08-11T15:44:25.184424+0000","last_peered":"2023-08-11T15:44:25.184424+0000","last_clean":"2023-08-11T15:44:25.184424+0000","last_became_active":"2023-08-11T15:44:24.197181+0000","last_became_peered":"2023-08-11T15:44:24.197181+0000","last_unstale":"2023-08-11T15:44:25.184424+0000","last_undegraded":"2023-08-11T15:44:25.184424+0000","last_fullsized":"2023-08-11T15:44:25.184424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:23:45.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043675+0000","last_change":"2023-08-11T15:44:19.147550+0000","last_active":"2023-08-11T15:44:27.043675+0000","last_peered":"2023-08-11T15:44:27.043675+0000","last_clean":"2023-08-11T15:44:27.043675+0000","last_became_active":"2023-08-11T15:44:19.147299+0000","last_became_peered":"2023-08-11T15:44:19.147299+0000","last_unstale":"2023-08-11T15:44:27.043675+0000","last_undegraded":"2023-08-11T15:44:27.043675+0000","last_fullsized":"2023-08-11T15:44:27.043675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:56.050406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189968+0000","last_change":"2023-08-11T15:44:19.862664+0000","last_active":"2023-08-11T15:44:54.189968+0000","last_peered":"2023-08-11T15:44:54.189968+0000","last_clean":"2023-08-11T15:44:54.189968+0000","last_became_active":"2023-08-11T15:44:19.862130+0000","last_became_peered":"2023-08-11T15:44:19.862130+0000","last_unstale":"2023-08-11T15:44:54.189968+0000","last_undegraded":"2023-08-11T15:44:54.189968+0000","last_fullsized":"2023-08-11T15:44:54.189968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:02.591318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181786+0000","last_change":"2023-08-11T15:44:24.199851+0000","last_active":"2023-08-11T15:44:25.181786+0000","last_peered":"2023-08-11T15:44:25.181786+0000","last_clean":"2023-08-11T15:44:25.181786+0000","last_became_active":"2023-08-11T15:44:24.199420+0000","last_became_peered":"2023-08-11T15:44:24.199420+0000","last_unstale":"2023-08-11T15:44:25.181786+0000","last_undegraded":"2023-08-11T15:44:25.181786+0000","last_fullsized":"2023-08-11T15:44:25.181786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:36:31.272229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196088+0000","last_change":"2023-08-11T15:44:19.145888+0000","last_active":"2023-08-11T15:44:25.196088+0000","last_peered":"2023-08-11T15:44:25.196088+0000","last_clean":"2023-08-11T15:44:25.196088+0000","last_became_active":"2023-08-11T15:44:19.145459+0000","last_became_peered":"2023-08-11T15:44:19.145459+0000","last_unstale":"2023-08-11T15:44:25.196088+0000","last_undegraded":"2023-08-11T15:44:25.196088+0000","last_fullsized":"2023-08-11T15:44:25.196088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:36.387374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189722+0000","last_change":"2023-08-11T15:44:19.860035+0000","last_active":"2023-08-11T15:44:54.189722+0000","last_peered":"2023-08-11T15:44:54.189722+0000","last_clean":"2023-08-11T15:44:54.189722+0000","last_became_active":"2023-08-11T15:44:19.859696+0000","last_became_peered":"2023-08-11T15:44:19.859696+0000","last_unstale":"2023-08-11T15:44:54.189722+0000","last_undegraded":"2023-08-11T15:44:54.189722+0000","last_fullsized":"2023-08-11T15:44:54.189722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:24.878953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399086+0000","last_change":"2023-08-11T15:44:24.198255+0000","last_active":"2023-08-11T15:44:25.399086+0000","last_peered":"2023-08-11T15:44:25.399086+0000","last_clean":"2023-08-11T15:44:25.399086+0000","last_became_active":"2023-08-11T15:44:24.197825+0000","last_became_peered":"2023-08-11T15:44:24.197825+0000","last_unstale":"2023-08-11T15:44:25.399086+0000","last_undegraded":"2023-08-11T15:44:25.399086+0000","last_fullsized":"2023-08-11T15:44:25.399086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:41:49.976978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496775+0000","last_change":"2023-08-11T15:44:19.151410+0000","last_active":"2023-08-11T15:44:27.496775+0000","last_peered":"2023-08-11T15:44:27.496775+0000","last_clean":"2023-08-11T15:44:27.496775+0000","last_became_active":"2023-08-11T15:44:19.151117+0000","last_became_peered":"2023-08-11T15:44:19.151117+0000","last_unstale":"2023-08-11T15:44:27.496775+0000","last_undegraded":"2023-08-11T15:44:27.496775+0000","last_fullsized":"2023-08-11T15:44:27.496775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:11.127495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043467+0000","last_change":"2023-08-11T15:44:19.853273+0000","last_active":"2023-08-11T15:44:27.043467+0000","last_peered":"2023-08-11T15:44:27.043467+0000","last_clean":"2023-08-11T15:44:27.043467+0000","last_became_active":"2023-08-11T15:44:19.852914+0000","last_became_peered":"2023-08-11T15:44:19.852914+0000","last_unstale":"2023-08-11T15:44:27.043467+0000","last_undegraded":"2023-08-11T15:44:27.043467+0000","last_fullsized":"2023-08-11T15:44:27.043467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:45.110625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043436+0000","last_change":"2023-08-11T15:44:24.194429+0000","last_active":"2023-08-11T15:44:27.043436+0000","last_peered":"2023-08-11T15:44:27.043436+0000","last_clean":"2023-08-11T15:44:27.043436+0000","last_became_active":"2023-08-11T15:44:24.194089+0000","last_became_peered":"2023-08-11T15:44:24.194089+0000","last_unstale":"2023-08-11T15:44:27.043436+0000","last_undegraded":"2023-08-11T15:44:27.043436+0000","last_fullsized":"2023-08-11T15:44:27.043436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:53.530309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194680+0000","last_change":"2023-08-11T15:44:19.151072+0000","last_active":"2023-08-11T15:44:54.194680+0000","last_peered":"2023-08-11T15:44:54.194680+0000","last_clean":"2023-08-11T15:44:54.194680+0000","last_became_active":"2023-08-11T15:44:19.147713+0000","last_became_peered":"2023-08-11T15:44:19.147713+0000","last_unstale":"2023-08-11T15:44:54.194680+0000","last_undegraded":"2023-08-11T15:44:54.194680+0000","last_fullsized":"2023-08-11T15:44:54.194680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:09:42.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184182+0000","last_change":"2023-08-11T15:44:19.862259+0000","last_active":"2023-08-11T15:44:25.184182+0000","last_peered":"2023-08-11T15:44:25.184182+0000","last_clean":"2023-08-11T15:44:25.184182+0000","last_became_active":"2023-08-11T15:44:19.861910+0000","last_became_peered":"2023-08-11T15:44:19.861910+0000","last_unstale":"2023-08-11T15:44:25.184182+0000","last_undegraded":"2023-08-11T15:44:25.184182+0000","last_fullsized":"2023-08-11T15:44:25.184182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:06:41.610675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.497633+0000","last_change":"2023-08-11T15:44:54.195412+0000","last_active":"2023-08-11T15:44:54.195412+0000","last_peered":"2023-08-11T15:44:27.497633+0000","last_clean":"2023-08-11T15:44:27.497633+0000","last_became_active":"2023-08-11T15:44:24.300268+0000","last_became_peered":"2023-08-11T15:44:24.300268+0000","last_unstale":"2023-08-11T15:44:27.497633+0000","last_undegraded":"2023-08-11T15:44:27.497633+0000","last_fullsized":"2023-08-11T15:44:27.497633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181579+0000","last_change":"2023-08-11T15:44:19.152608+0000","last_active":"2023-08-11T15:44:25.181579+0000","last_peered":"2023-08-11T15:44:25.181579+0000","last_clean":"2023-08-11T15:44:25.181579+0000","last_became_active":"2023-08-11T15:44:19.152255+0000","last_became_peered":"2023-08-11T15:44:19.152255+0000","last_unstale":"2023-08-11T15:44:25.181579+0000","last_undegraded":"2023-08-11T15:44:25.181579+0000","last_fullsized":"2023-08-11T15:44:25.181579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:33.299760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909653+0000","last_change":"2023-08-11T15:44:19.864003+0000","last_active":"2023-08-11T15:44:26.909653+0000","last_peered":"2023-08-11T15:44:26.909653+0000","last_clean":"2023-08-11T15:44:26.909653+0000","last_became_active":"2023-08-11T15:44:19.863661+0000","last_became_peered":"2023-08-11T15:44:19.863661+0000","last_unstale":"2023-08-11T15:44:26.909653+0000","last_undegraded":"2023-08-11T15:44:26.909653+0000","last_fullsized":"2023-08-11T15:44:26.909653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:20:35.118164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398869+0000","last_change":"2023-08-11T15:44:24.194123+0000","last_active":"2023-08-11T15:44:25.398869+0000","last_peered":"2023-08-11T15:44:25.398869+0000","last_clean":"2023-08-11T15:44:25.398869+0000","last_became_active":"2023-08-11T15:44:24.193729+0000","last_became_peered":"2023-08-11T15:44:24.193729+0000","last_unstale":"2023-08-11T15:44:25.398869+0000","last_undegraded":"2023-08-11T15:44:25.398869+0000","last_fullsized":"2023-08-11T15:44:25.398869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:11:10.722179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189631+0000","last_change":"2023-08-11T15:44:19.151367+0000","last_active":"2023-08-11T15:44:54.189631+0000","last_peered":"2023-08-11T15:44:54.189631+0000","last_clean":"2023-08-11T15:44:54.189631+0000","last_became_active":"2023-08-11T15:44:19.150991+0000","last_became_peered":"2023-08-11T15:44:19.150991+0000","last_unstale":"2023-08-11T15:44:54.189631+0000","last_undegraded":"2023-08-11T15:44:54.189631+0000","last_fullsized":"2023-08-11T15:44:54.189631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:03:03.249036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909318+0000","last_change":"2023-08-11T15:44:19.858579+0000","last_active":"2023-08-11T15:44:26.909318+0000","last_peered":"2023-08-11T15:44:26.909318+0000","last_clean":"2023-08-11T15:44:26.909318+0000","last_became_active":"2023-08-11T15:44:19.858184+0000","last_became_peered":"2023-08-11T15:44:19.858184+0000","last_unstale":"2023-08-11T15:44:26.909318+0000","last_undegraded":"2023-08-11T15:44:26.909318+0000","last_fullsized":"2023-08-11T15:44:26.909318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:38.034222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399074+0000","last_change":"2023-08-11T15:44:24.300526+0000","last_active":"2023-08-11T15:44:25.399074+0000","last_peered":"2023-08-11T15:44:25.399074+0000","last_clean":"2023-08-11T15:44:25.399074+0000","last_became_active":"2023-08-11T15:44:24.299883+0000","last_became_peered":"2023-08-11T15:44:24.299883+0000","last_unstale":"2023-08-11T15:44:25.399074+0000","last_undegraded":"2023-08-11T15:44:25.399074+0000","last_fullsized":"2023-08-11T15:44:25.399074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:53:45.487528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043468+0000","last_change":"2023-08-11T15:44:19.858116+0000","last_active":"2023-08-11T15:44:27.043468+0000","last_peered":"2023-08-11T15:44:27.043468+0000","last_clean":"2023-08-11T15:44:27.043468+0000","last_became_active":"2023-08-11T15:44:19.857728+0000","last_became_peered":"2023-08-11T15:44:19.857728+0000","last_unstale":"2023-08-11T15:44:27.043468+0000","last_undegraded":"2023-08-11T15:44:27.043468+0000","last_fullsized":"2023-08-11T15:44:27.043468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:56:36.953948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398236+0000","last_change":"2023-08-11T15:44:19.157904+0000","last_active":"2023-08-11T15:44:25.398236+0000","last_peered":"2023-08-11T15:44:25.398236+0000","last_clean":"2023-08-11T15:44:25.398236+0000","last_became_active":"2023-08-11T15:44:19.157497+0000","last_became_peered":"2023-08-11T15:44:19.157497+0000","last_unstale":"2023-08-11T15:44:25.398236+0000","last_undegraded":"2023-08-11T15:44:25.398236+0000","last_fullsized":"2023-08-11T15:44:25.398236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:10.337630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189839+0000","last_change":"2023-08-11T15:44:24.201246+0000","last_active":"2023-08-11T15:44:54.189839+0000","last_peered":"2023-08-11T15:44:54.189839+0000","last_clean":"2023-08-11T15:44:54.189839+0000","last_became_active":"2023-08-11T15:44:24.200810+0000","last_became_peered":"2023-08-11T15:44:24.200810+0000","last_unstale":"2023-08-11T15:44:54.189839+0000","last_undegraded":"2023-08-11T15:44:54.189839+0000","last_fullsized":"2023-08-11T15:44:54.189839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:47:27.192243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178569+0000","last_change":"2023-08-11T15:44:19.846296+0000","last_active":"2023-08-11T15:44:25.178569+0000","last_peered":"2023-08-11T15:44:25.178569+0000","last_clean":"2023-08-11T15:44:25.178569+0000","last_became_active":"2023-08-11T15:44:19.845890+0000","last_became_peered":"2023-08-11T15:44:19.845890+0000","last_unstale":"2023-08-11T15:44:25.178569+0000","last_undegraded":"2023-08-11T15:44:25.178569+0000","last_fullsized":"2023-08-11T15:44:25.178569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:44:22.834705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196213+0000","last_change":"2023-08-11T15:44:19.144202+0000","last_active":"2023-08-11T15:44:25.196213+0000","last_peered":"2023-08-11T15:44:25.196213+0000","last_clean":"2023-08-11T15:44:25.196213+0000","last_became_active":"2023-08-11T15:44:19.143789+0000","last_became_peered":"2023-08-11T15:44:19.143789+0000","last_unstale":"2023-08-11T15:44:25.196213+0000","last_undegraded":"2023-08-11T15:44:25.196213+0000","last_fullsized":"2023-08-11T15:44:25.196213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:57:56.919463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189289+0000","last_change":"2023-08-11T15:43:14.593634+0000","last_active":"2023-08-11T15:44:54.189289+0000","last_peered":"2023-08-11T15:44:54.189289+0000","last_clean":"2023-08-11T15:44:54.189289+0000","last_became_active":"2023-08-11T15:43:14.593164+0000","last_became_peered":"2023-08-11T15:43:14.593164+0000","last_unstale":"2023-08-11T15:44:54.189289+0000","last_undegraded":"2023-08-11T15:44:54.189289+0000","last_fullsized":"2023-08-11T15:44:54.189289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_clean_scrub_stamp":"2023-08-11T15:40:09.009169+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:15.991425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044055+0000","last_change":"2023-08-11T15:44:24.300703+0000","last_active":"2023-08-11T15:44:27.044055+0000","last_peered":"2023-08-11T15:44:27.044055+0000","last_clean":"2023-08-11T15:44:27.044055+0000","last_became_active":"2023-08-11T15:44:24.300309+0000","last_became_peered":"2023-08-11T15:44:24.300309+0000","last_unstale":"2023-08-11T15:44:27.044055+0000","last_undegraded":"2023-08-11T15:44:27.044055+0000","last_fullsized":"2023-08-11T15:44:27.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:41.018379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398223+0000","last_change":"2023-08-11T15:44:19.858934+0000","last_active":"2023-08-11T15:44:25.398223+0000","last_peered":"2023-08-11T15:44:25.398223+0000","last_clean":"2023-08-11T15:44:25.398223+0000","last_became_active":"2023-08-11T15:44:19.858613+0000","last_became_peered":"2023-08-11T15:44:19.858613+0000","last_unstale":"2023-08-11T15:44:25.398223+0000","last_undegraded":"2023-08-11T15:44:25.398223+0000","last_fullsized":"2023-08-11T15:44:25.398223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:07.938228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189544+0000","last_change":"2023-08-11T15:44:19.148113+0000","last_active":"2023-08-11T15:44:54.189544+0000","last_peered":"2023-08-11T15:44:54.189544+0000","last_clean":"2023-08-11T15:44:54.189544+0000","last_became_active":"2023-08-11T15:44:19.147793+0000","last_became_peered":"2023-08-11T15:44:19.147793+0000","last_unstale":"2023-08-11T15:44:54.189544+0000","last_undegraded":"2023-08-11T15:44:54.189544+0000","last_fullsized":"2023-08-11T15:44:54.189544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:07:38.923167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184402+0000","last_change":"2023-08-11T15:44:24.194418+0000","last_active":"2023-08-11T15:44:25.184402+0000","last_peered":"2023-08-11T15:44:25.184402+0000","last_clean":"2023-08-11T15:44:25.184402+0000","last_became_active":"2023-08-11T15:44:24.193943+0000","last_became_peered":"2023-08-11T15:44:24.193943+0000","last_unstale":"2023-08-11T15:44:25.184402+0000","last_undegraded":"2023-08-11T15:44:25.184402+0000","last_fullsized":"2023-08-11T15:44:25.184402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:35.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196145+0000","last_change":"2023-08-11T15:44:19.151476+0000","last_active":"2023-08-11T15:44:54.196145+0000","last_peered":"2023-08-11T15:44:54.196145+0000","last_clean":"2023-08-11T15:44:54.196145+0000","last_became_active":"2023-08-11T15:44:19.151140+0000","last_became_peered":"2023-08-11T15:44:19.151140+0000","last_unstale":"2023-08-11T15:44:54.196145+0000","last_undegraded":"2023-08-11T15:44:54.196145+0000","last_fullsized":"2023-08-11T15:44:54.196145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:08:09.404210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398895+0000","last_change":"2023-08-11T15:44:19.862255+0000","last_active":"2023-08-11T15:44:25.398895+0000","last_peered":"2023-08-11T15:44:25.398895+0000","last_clean":"2023-08-11T15:44:25.398895+0000","last_became_active":"2023-08-11T15:44:19.861907+0000","last_became_peered":"2023-08-11T15:44:19.861907+0000","last_unstale":"2023-08-11T15:44:25.398895+0000","last_undegraded":"2023-08-11T15:44:25.398895+0000","last_fullsized":"2023-08-11T15:44:25.398895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:03.436493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178469+0000","last_change":"2023-08-11T15:44:24.300513+0000","last_active":"2023-08-11T15:44:25.178469+0000","last_peered":"2023-08-11T15:44:25.178469+0000","last_clean":"2023-08-11T15:44:25.178469+0000","last_became_active":"2023-08-11T15:44:24.300074+0000","last_became_peered":"2023-08-11T15:44:24.300074+0000","last_unstale":"2023-08-11T15:44:25.178469+0000","last_undegraded":"2023-08-11T15:44:25.178469+0000","last_fullsized":"2023-08-11T15:44:25.178469+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:16:21.212112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181835+0000","last_change":"2023-08-11T15:44:19.153794+0000","last_active":"2023-08-11T15:44:25.181835+0000","last_peered":"2023-08-11T15:44:25.181835+0000","last_clean":"2023-08-11T15:44:25.181835+0000","last_became_active":"2023-08-11T15:44:19.153392+0000","last_became_peered":"2023-08-11T15:44:19.153392+0000","last_unstale":"2023-08-11T15:44:25.181835+0000","last_undegraded":"2023-08-11T15:44:25.181835+0000","last_fullsized":"2023-08-11T15:44:25.181835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:47.991902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179190+0000","last_change":"2023-08-11T15:44:19.859799+0000","last_active":"2023-08-11T15:44:25.179190+0000","last_peered":"2023-08-11T15:44:25.179190+0000","last_clean":"2023-08-11T15:44:25.179190+0000","last_became_active":"2023-08-11T15:44:19.859411+0000","last_became_peered":"2023-08-11T15:44:19.859411+0000","last_unstale":"2023-08-11T15:44:25.179190+0000","last_undegraded":"2023-08-11T15:44:25.179190+0000","last_fullsized":"2023-08-11T15:44:25.179190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:19.443644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130327+0000","last_change":"2023-08-11T15:44:24.190272+0000","last_active":"2023-08-11T15:44:27.130327+0000","last_peered":"2023-08-11T15:44:27.130327+0000","last_clean":"2023-08-11T15:44:27.130327+0000","last_became_active":"2023-08-11T15:44:24.189849+0000","last_became_peered":"2023-08-11T15:44:24.189849+0000","last_unstale":"2023-08-11T15:44:27.130327+0000","last_undegraded":"2023-08-11T15:44:27.130327+0000","last_fullsized":"2023-08-11T15:44:27.130327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:26:26.603082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196757+0000","last_change":"2023-08-11T15:44:19.154985+0000","last_active":"2023-08-11T15:44:25.196757+0000","last_peered":"2023-08-11T15:44:25.196757+0000","last_clean":"2023-08-11T15:44:25.196757+0000","last_became_active":"2023-08-11T15:44:19.154694+0000","last_became_peered":"2023-08-11T15:44:19.154694+0000","last_unstale":"2023-08-11T15:44:25.196757+0000","last_undegraded":"2023-08-11T15:44:25.196757+0000","last_fullsized":"2023-08-11T15:44:25.196757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:15.737671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241127+0000","last_change":"2023-08-11T15:44:19.852247+0000","last_active":"2023-08-11T15:44:26.241127+0000","last_peered":"2023-08-11T15:44:26.241127+0000","last_clean":"2023-08-11T15:44:26.241127+0000","last_became_active":"2023-08-11T15:44:19.851456+0000","last_became_peered":"2023-08-11T15:44:19.851456+0000","last_unstale":"2023-08-11T15:44:26.241127+0000","last_undegraded":"2023-08-11T15:44:26.241127+0000","last_fullsized":"2023-08-11T15:44:26.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:57:45.705759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196509+0000","last_change":"2023-08-11T15:44:24.300440+0000","last_active":"2023-08-11T15:44:25.196509+0000","last_peered":"2023-08-11T15:44:25.196509+0000","last_clean":"2023-08-11T15:44:25.196509+0000","last_became_active":"2023-08-11T15:44:24.299981+0000","last_became_peered":"2023-08-11T15:44:24.299981+0000","last_unstale":"2023-08-11T15:44:25.196509+0000","last_undegraded":"2023-08-11T15:44:25.196509+0000","last_fullsized":"2023-08-11T15:44:25.196509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:24.190521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496438+0000","last_change":"2023-08-11T15:44:19.151015+0000","last_active":"2023-08-11T15:44:27.496438+0000","last_peered":"2023-08-11T15:44:27.496438+0000","last_clean":"2023-08-11T15:44:27.496438+0000","last_became_active":"2023-08-11T15:44:19.150677+0000","last_became_peered":"2023-08-11T15:44:19.150677+0000","last_unstale":"2023-08-11T15:44:27.496438+0000","last_undegraded":"2023-08-11T15:44:27.496438+0000","last_fullsized":"2023-08-11T15:44:27.496438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:38:34.368253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.195188+0000","last_change":"2023-08-11T15:44:19.861033+0000","last_active":"2023-08-11T15:44:54.195188+0000","last_peered":"2023-08-11T15:44:54.195188+0000","last_clean":"2023-08-11T15:44:54.195188+0000","last_became_active":"2023-08-11T15:44:19.860643+0000","last_became_peered":"2023-08-11T15:44:19.860643+0000","last_unstale":"2023-08-11T15:44:54.195188+0000","last_undegraded":"2023-08-11T15:44:54.195188+0000","last_fullsized":"2023-08-11T15:44:54.195188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:56:51.466526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181930+0000","last_change":"2023-08-11T15:44:24.199389+0000","last_active":"2023-08-11T15:44:25.181930+0000","last_peered":"2023-08-11T15:44:25.181930+0000","last_clean":"2023-08-11T15:44:25.181930+0000","last_became_active":"2023-08-11T15:44:24.196226+0000","last_became_peered":"2023-08-11T15:44:24.196226+0000","last_unstale":"2023-08-11T15:44:25.181930+0000","last_undegraded":"2023-08-11T15:44:25.181930+0000","last_fullsized":"2023-08-11T15:44:25.181930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:03.253768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178850+0000","last_change":"2023-08-11T15:44:19.152895+0000","last_active":"2023-08-11T15:44:25.178850+0000","last_peered":"2023-08-11T15:44:25.178850+0000","last_clean":"2023-08-11T15:44:25.178850+0000","last_became_active":"2023-08-11T15:44:19.152577+0000","last_became_peered":"2023-08-11T15:44:19.152577+0000","last_unstale":"2023-08-11T15:44:25.178850+0000","last_undegraded":"2023-08-11T15:44:25.178850+0000","last_fullsized":"2023-08-11T15:44:25.178850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:41:12.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183940+0000","last_change":"2023-08-11T15:44:19.861101+0000","last_active":"2023-08-11T15:44:25.183940+0000","last_peered":"2023-08-11T15:44:25.183940+0000","last_clean":"2023-08-11T15:44:25.183940+0000","last_became_active":"2023-08-11T15:44:19.860529+0000","last_became_peered":"2023-08-11T15:44:19.860529+0000","last_unstale":"2023-08-11T15:44:25.183940+0000","last_undegraded":"2023-08-11T15:44:25.183940+0000","last_fullsized":"2023-08-11T15:44:25.183940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:25.463560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196480+0000","last_change":"2023-08-11T15:44:24.202856+0000","last_active":"2023-08-11T15:44:25.196480+0000","last_peered":"2023-08-11T15:44:25.196480+0000","last_clean":"2023-08-11T15:44:25.196480+0000","last_became_active":"2023-08-11T15:44:24.202317+0000","last_became_peered":"2023-08-11T15:44:24.202317+0000","last_unstale":"2023-08-11T15:44:25.196480+0000","last_undegraded":"2023-08-11T15:44:25.196480+0000","last_fullsized":"2023-08-11T15:44:25.196480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:12:44.969687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130438+0000","last_change":"2023-08-11T15:44:19.148711+0000","last_active":"2023-08-11T15:44:27.130438+0000","last_peered":"2023-08-11T15:44:27.130438+0000","last_clean":"2023-08-11T15:44:27.130438+0000","last_became_active":"2023-08-11T15:44:19.148361+0000","last_became_peered":"2023-08-11T15:44:19.148361+0000","last_unstale":"2023-08-11T15:44:27.130438+0000","last_undegraded":"2023-08-11T15:44:27.130438+0000","last_fullsized":"2023-08-11T15:44:27.130438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:45.703207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178814+0000","last_change":"2023-08-11T15:44:19.859562+0000","last_active":"2023-08-11T15:44:25.178814+0000","last_peered":"2023-08-11T15:44:25.178814+0000","last_clean":"2023-08-11T15:44:25.178814+0000","last_became_active":"2023-08-11T15:44:19.859243+0000","last_became_peered":"2023-08-11T15:44:19.859243+0000","last_unstale":"2023-08-11T15:44:25.178814+0000","last_undegraded":"2023-08-11T15:44:25.178814+0000","last_fullsized":"2023-08-11T15:44:25.178814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:50.690599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196555+0000","last_change":"2023-08-11T15:44:24.300317+0000","last_active":"2023-08-11T15:44:25.196555+0000","last_peered":"2023-08-11T15:44:25.196555+0000","last_clean":"2023-08-11T15:44:25.196555+0000","last_became_active":"2023-08-11T15:44:24.299841+0000","last_became_peered":"2023-08-11T15:44:24.299841+0000","last_unstale":"2023-08-11T15:44:25.196555+0000","last_undegraded":"2023-08-11T15:44:25.196555+0000","last_fullsized":"2023-08-11T15:44:25.196555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:16.800305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194494+0000","last_change":"2023-08-11T15:44:19.149111+0000","last_active":"2023-08-11T15:44:54.194494+0000","last_peered":"2023-08-11T15:44:54.194494+0000","last_clean":"2023-08-11T15:44:54.194494+0000","last_became_active":"2023-08-11T15:44:19.148173+0000","last_became_peered":"2023-08-11T15:44:19.148173+0000","last_unstale":"2023-08-11T15:44:54.194494+0000","last_undegraded":"2023-08-11T15:44:54.194494+0000","last_fullsized":"2023-08-11T15:44:54.194494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:51.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189482+0000","last_change":"2023-08-11T15:44:19.863724+0000","last_active":"2023-08-11T15:44:54.189482+0000","last_peered":"2023-08-11T15:44:54.189482+0000","last_clean":"2023-08-11T15:44:54.189482+0000","last_became_active":"2023-08-11T15:44:19.863268+0000","last_became_peered":"2023-08-11T15:44:19.863268+0000","last_unstale":"2023-08-11T15:44:54.189482+0000","last_undegraded":"2023-08-11T15:44:54.189482+0000","last_fullsized":"2023-08-11T15:44:54.189482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:27:34.481889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497528+0000","last_change":"2023-08-11T15:44:24.191301+0000","last_active":"2023-08-11T15:44:27.497528+0000","last_peered":"2023-08-11T15:44:27.497528+0000","last_clean":"2023-08-11T15:44:27.497528+0000","last_became_active":"2023-08-11T15:44:24.190686+0000","last_became_peered":"2023-08-11T15:44:24.190686+0000","last_unstale":"2023-08-11T15:44:27.497528+0000","last_undegraded":"2023-08-11T15:44:27.497528+0000","last_fullsized":"2023-08-11T15:44:27.497528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:29:52.896841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130772+0000","last_change":"2023-08-11T15:44:19.147756+0000","last_active":"2023-08-11T15:44:27.130772+0000","last_peered":"2023-08-11T15:44:27.130772+0000","last_clean":"2023-08-11T15:44:27.130772+0000","last_became_active":"2023-08-11T15:44:19.147426+0000","last_became_peered":"2023-08-11T15:44:19.147426+0000","last_unstale":"2023-08-11T15:44:27.130772+0000","last_undegraded":"2023-08-11T15:44:27.130772+0000","last_fullsized":"2023-08-11T15:44:27.130772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:45.863313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190289+0000","last_change":"2023-08-11T15:44:19.863354+0000","last_active":"2023-08-11T15:44:54.190289+0000","last_peered":"2023-08-11T15:44:54.190289+0000","last_clean":"2023-08-11T15:44:54.190289+0000","last_became_active":"2023-08-11T15:44:19.862700+0000","last_became_peered":"2023-08-11T15:44:19.862700+0000","last_unstale":"2023-08-11T15:44:54.190289+0000","last_undegraded":"2023-08-11T15:44:54.190289+0000","last_fullsized":"2023-08-11T15:44:54.190289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:14:44.778310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240828+0000","last_change":"2023-08-11T15:44:24.204657+0000","last_active":"2023-08-11T15:44:26.240828+0000","last_peered":"2023-08-11T15:44:26.240828+0000","last_clean":"2023-08-11T15:44:26.240828+0000","last_became_active":"2023-08-11T15:44:24.204108+0000","last_became_peered":"2023-08-11T15:44:24.204108+0000","last_unstale":"2023-08-11T15:44:26.240828+0000","last_undegraded":"2023-08-11T15:44:26.240828+0000","last_fullsized":"2023-08-11T15:44:26.240828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:17.431253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044145+0000","last_change":"2023-08-11T15:44:19.147014+0000","last_active":"2023-08-11T15:44:27.044145+0000","last_peered":"2023-08-11T15:44:27.044145+0000","last_clean":"2023-08-11T15:44:27.044145+0000","last_became_active":"2023-08-11T15:44:19.146720+0000","last_became_peered":"2023-08-11T15:44:19.146720+0000","last_unstale":"2023-08-11T15:44:27.044145+0000","last_undegraded":"2023-08-11T15:44:27.044145+0000","last_fullsized":"2023-08-11T15:44:27.044145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:42:56.620766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.498380+0000","last_change":"2023-08-11T15:44:54.190492+0000","last_active":"2023-08-11T15:44:54.190492+0000","last_peered":"2023-08-11T15:44:27.498380+0000","last_clean":"2023-08-11T15:44:27.498380+0000","last_became_active":"2023-08-11T15:44:19.853078+0000","last_became_peered":"2023-08-11T15:44:19.853078+0000","last_unstale":"2023-08-11T15:44:27.498380+0000","last_undegraded":"2023-08-11T15:44:27.498380+0000","last_fullsized":"2023-08-11T15:44:27.498380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043195+0000","last_change":"2023-08-11T15:44:24.192709+0000","last_active":"2023-08-11T15:44:27.043195+0000","last_peered":"2023-08-11T15:44:27.043195+0000","last_clean":"2023-08-11T15:44:27.043195+0000","last_became_active":"2023-08-11T15:44:24.192297+0000","last_became_peered":"2023-08-11T15:44:24.192297+0000","last_unstale":"2023-08-11T15:44:27.043195+0000","last_undegraded":"2023-08-11T15:44:27.043195+0000","last_fullsized":"2023-08-11T15:44:27.043195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:39.723615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399018+0000","last_change":"2023-08-11T15:44:19.158301+0000","last_active":"2023-08-11T15:44:25.399018+0000","last_peered":"2023-08-11T15:44:25.399018+0000","last_clean":"2023-08-11T15:44:25.399018+0000","last_became_active":"2023-08-11T15:44:19.157990+0000","last_became_peered":"2023-08-11T15:44:19.157990+0000","last_unstale":"2023-08-11T15:44:25.399018+0000","last_undegraded":"2023-08-11T15:44:25.399018+0000","last_fullsized":"2023-08-11T15:44:25.399018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:02.696501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189944+0000","last_change":"2023-08-11T15:44:19.857357+0000","last_active":"2023-08-11T15:44:54.189944+0000","last_peered":"2023-08-11T15:44:54.189944+0000","last_clean":"2023-08-11T15:44:54.189944+0000","last_became_active":"2023-08-11T15:44:19.857020+0000","last_became_peered":"2023-08-11T15:44:19.857020+0000","last_unstale":"2023-08-11T15:44:54.189944+0000","last_undegraded":"2023-08-11T15:44:54.189944+0000","last_fullsized":"2023-08-11T15:44:54.189944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:37.243586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183978+0000","last_change":"2023-08-11T15:44:19.863314+0000","last_active":"2023-08-11T15:44:25.183978+0000","last_peered":"2023-08-11T15:44:25.183978+0000","last_clean":"2023-08-11T15:44:25.183978+0000","last_became_active":"2023-08-11T15:44:19.862964+0000","last_became_peered":"2023-08-11T15:44:19.862964+0000","last_unstale":"2023-08-11T15:44:25.183978+0000","last_undegraded":"2023-08-11T15:44:25.183978+0000","last_fullsized":"2023-08-11T15:44:25.183978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:16.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130670+0000","last_change":"2023-08-11T15:44:19.850575+0000","last_active":"2023-08-11T15:44:27.130670+0000","last_peered":"2023-08-11T15:44:27.130670+0000","last_clean":"2023-08-11T15:44:27.130670+0000","last_became_active":"2023-08-11T15:44:19.850231+0000","last_became_peered":"2023-08-11T15:44:19.850231+0000","last_unstale":"2023-08-11T15:44:27.130670+0000","last_undegraded":"2023-08-11T15:44:27.130670+0000","last_fullsized":"2023-08-11T15:44:27.130670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:21.005350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130303+0000","last_change":"2023-08-11T15:44:19.857387+0000","last_active":"2023-08-11T15:44:27.130303+0000","last_peered":"2023-08-11T15:44:27.130303+0000","last_clean":"2023-08-11T15:44:27.130303+0000","last_became_active":"2023-08-11T15:44:19.856830+0000","last_became_peered":"2023-08-11T15:44:19.856830+0000","last_unstale":"2023-08-11T15:44:27.130303+0000","last_undegraded":"2023-08-11T15:44:27.130303+0000","last_fullsized":"2023-08-11T15:44:27.130303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:42.512448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241040+0000","last_change":"2023-08-11T15:44:19.868112+0000","last_active":"2023-08-11T15:44:26.241040+0000","last_peered":"2023-08-11T15:44:26.241040+0000","last_clean":"2023-08-11T15:44:26.241040+0000","last_became_active":"2023-08-11T15:44:19.867610+0000","last_became_peered":"2023-08-11T15:44:19.867610+0000","last_unstale":"2023-08-11T15:44:26.241040+0000","last_undegraded":"2023-08-11T15:44:26.241040+0000","last_fullsized":"2023-08-11T15:44:26.241040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:18.810119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398997+0000","last_change":"2023-08-11T15:44:19.858349+0000","last_active":"2023-08-11T15:44:25.398997+0000","last_peered":"2023-08-11T15:44:25.398997+0000","last_clean":"2023-08-11T15:44:25.398997+0000","last_became_active":"2023-08-11T15:44:19.858041+0000","last_became_peered":"2023-08-11T15:44:19.858041+0000","last_unstale":"2023-08-11T15:44:25.398997+0000","last_undegraded":"2023-08-11T15:44:25.398997+0000","last_fullsized":"2023-08-11T15:44:25.398997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:04.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184464+0000","last_change":"2023-08-11T15:44:19.865383+0000","last_active":"2023-08-11T15:44:25.184464+0000","last_peered":"2023-08-11T15:44:25.184464+0000","last_clean":"2023-08-11T15:44:25.184464+0000","last_became_active":"2023-08-11T15:44:19.865065+0000","last_became_peered":"2023-08-11T15:44:19.865065+0000","last_unstale":"2023-08-11T15:44:25.184464+0000","last_undegraded":"2023-08-11T15:44:25.184464+0000","last_fullsized":"2023-08-11T15:44:25.184464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:39:27.425778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130961+0000","last_change":"2023-08-11T15:44:19.859297+0000","last_active":"2023-08-11T15:44:27.130961+0000","last_peered":"2023-08-11T15:44:27.130961+0000","last_clean":"2023-08-11T15:44:27.130961+0000","last_became_active":"2023-08-11T15:44:19.857517+0000","last_became_peered":"2023-08-11T15:44:19.857517+0000","last_unstale":"2023-08-11T15:44:27.130961+0000","last_undegraded":"2023-08-11T15:44:27.130961+0000","last_fullsized":"2023-08-11T15:44:27.130961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:52:53.104396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179277+0000","last_change":"2023-08-11T15:44:19.850485+0000","last_active":"2023-08-11T15:44:25.179277+0000","last_peered":"2023-08-11T15:44:25.179277+0000","last_clean":"2023-08-11T15:44:25.179277+0000","last_became_active":"2023-08-11T15:44:19.850047+0000","last_became_peered":"2023-08-11T15:44:19.850047+0000","last_unstale":"2023-08-11T15:44:25.179277+0000","last_undegraded":"2023-08-11T15:44:25.179277+0000","last_fullsized":"2023-08-11T15:44:25.179277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:11.380061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043361+0000","last_change":"2023-08-11T15:44:19.861953+0000","last_active":"2023-08-11T15:44:27.043361+0000","last_peered":"2023-08-11T15:44:27.043361+0000","last_clean":"2023-08-11T15:44:27.043361+0000","last_became_active":"2023-08-11T15:44:19.861523+0000","last_became_peered":"2023-08-11T15:44:19.861523+0000","last_unstale":"2023-08-11T15:44:27.043361+0000","last_undegraded":"2023-08-11T15:44:27.043361+0000","last_fullsized":"2023-08-11T15:44:27.043361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:14.229621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178755+0000","last_change":"2023-08-11T15:44:19.849283+0000","last_active":"2023-08-11T15:44:25.178755+0000","last_peered":"2023-08-11T15:44:25.178755+0000","last_clean":"2023-08-11T15:44:25.178755+0000","last_became_active":"2023-08-11T15:44:19.848304+0000","last_became_peered":"2023-08-11T15:44:19.848304+0000","last_unstale":"2023-08-11T15:44:25.178755+0000","last_undegraded":"2023-08-11T15:44:25.178755+0000","last_fullsized":"2023-08-11T15:44:25.178755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:06.449360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497606+0000","last_change":"2023-08-11T15:44:19.853532+0000","last_active":"2023-08-11T15:44:27.497606+0000","last_peered":"2023-08-11T15:44:27.497606+0000","last_clean":"2023-08-11T15:44:27.497606+0000","last_became_active":"2023-08-11T15:44:19.853182+0000","last_became_peered":"2023-08-11T15:44:19.853182+0000","last_unstale":"2023-08-11T15:44:27.497606+0000","last_undegraded":"2023-08-11T15:44:27.497606+0000","last_fullsized":"2023-08-11T15:44:27.497606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:51.245443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398519+0000","last_change":"2023-08-11T15:44:19.860571+0000","last_active":"2023-08-11T15:44:25.398519+0000","last_peered":"2023-08-11T15:44:25.398519+0000","last_clean":"2023-08-11T15:44:25.398519+0000","last_became_active":"2023-08-11T15:44:19.858570+0000","last_became_peered":"2023-08-11T15:44:19.858570+0000","last_unstale":"2023-08-11T15:44:25.398519+0000","last_undegraded":"2023-08-11T15:44:25.398519+0000","last_fullsized":"2023-08-11T15:44:25.398519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:37.343811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184137+0000","last_change":"2023-08-11T15:44:19.865988+0000","last_active":"2023-08-11T15:44:25.184137+0000","last_peered":"2023-08-11T15:44:25.184137+0000","last_clean":"2023-08-11T15:44:25.184137+0000","last_became_active":"2023-08-11T15:44:19.865613+0000","last_became_peered":"2023-08-11T15:44:19.865613+0000","last_unstale":"2023-08-11T15:44:25.184137+0000","last_undegraded":"2023-08-11T15:44:25.184137+0000","last_fullsized":"2023-08-11T15:44:25.184137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:07.472727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":48682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.77400000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1061196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.627}]},{"osd":3,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.503}]},{"osd":4,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38400000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.497}]},{"osd":8,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.61299999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83399999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.627},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.751}]},{"osd":6,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.67100000000000004}]},{"osd":8,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.80300000000000005}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":481002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.75800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72499999999999998,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.85799999999999998,"5min":1.05,"15min":1.05},"last":1.1160000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.435,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99199999999999999,"5min":1.077,"15min":1.077},"last":1.145},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0980000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.96899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95399999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":1.1859999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.5819999999999999,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":1.044}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.1299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1075775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.45500000000000002,"15min":0.44800000000000001},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.628},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Aug 11 15:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.79300000000000004,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.498},"max":{"1min":4.8890000000000002,"5min":4.8890000000000002,"15min":4.8890000000000002},"last":0.88}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":2.077,"15min":2.077},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.008,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.581,"15min":1.581},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92500000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.7,"15min":1.7},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0269999999999999,"5min":1.22,"15min":1.22},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1319999999999999,"5min":1.623,"15min":1.623},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.61799999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.63,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.90000000000000002,"15min":0.89200000000000002},"min":{"1min":0.64500000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.85499999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.98399999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.95199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":470965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.76600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88600000000000001,"5min":4.9020000000000001,"15min":4.9020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.45900000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.62}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80800000000000005,"5min":1.593,"15min":1.593},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 15:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.024,"15min":1.024},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74099999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":1.05,"15min":1.05},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.90100000000000002}]},{"osd":8,"last update":"Fri Aug 11 15:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67700000000000005,"15min":0.66600000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1180000000000001,"5min":1.298,"15min":1.298},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.5409999999999999,"5min":6.0609999999999999,"15min":6.0609999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.087,"5min":6.101,"15min":6.101},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.621,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.441,"15min":0.433},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46100000000000002,"15min":0.44700000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.65300000000000002,"15min":0.61299999999999999},"min":{"1min":0.56000000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.54300000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.056,"5min":1.175,"15min":1.175},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.032,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.74399999999999999,"15min":0.72799999999999998},"min":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.78900000000000003,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.85799999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1779999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.149,"5min":1.276,"15min":1.276},"last":1.0489999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.51400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.81999999999999995}]},{"osd":11,"last update":"Fri Aug 11 15:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.016}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":470999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54600000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80300000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.44},"min":{"1min":0.29699999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69499999999999995}]},{"osd":8,"last update":"Fri Aug 11 15:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.879},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.97499999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.95999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":467227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.47699999999999998,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68100000000000005,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.746}]},{"osd":2,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55600000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94399999999999995,"5min":1.105,"15min":1.105},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.53800000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.72399999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.58799999999999997,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Aug 11 15:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.629},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.73599999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95099999999999996,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.266,"5min":5.266,"15min":5.266},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.51000000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.84799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.317},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.35999999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83599999999999997,"5min":1.552,"15min":1.552},"last":0.626},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":1.502,"15min":1.502},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47499999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.016,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93899999999999995,"5min":1.526,"15min":1.526},"last":0.68999999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1057342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79500000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60599999999999998,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83299999999999996,"5min":5.359,"15min":5.359},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.434,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.872,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.443},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77000000000000002,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.36899999999999999},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41999999999999998,"15min":0.40400000000000003},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.73099999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.032,"15min":1.032},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82599999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]},{"osd":8,"last update":"Fri Aug 11 15:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.71599999999999997}]},{"osd":10,"last update":"Fri Aug 11 15:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":21257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:44:56.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:44:56.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 -- 172.21.15.32:0/3986695581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c100930 msgr2=0x7eff2c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 --2- 172.21.15.32:0/3986695581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c100930 0x7eff2c100d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7eff280099c0 tx=0x7eff28009cd0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 -- 172.21.15.32:0/3986695581 shutdown_connections 2023-08-11T15:44:56.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 --2- 172.21.15.32:0/3986695581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c100930 0x7eff2c100d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 --2- 172.21.15.32:0/3986695581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff2c1024c0 0x7eff2c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 --2- 172.21.15.32:0/3986695581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff2c101b20 0x7eff2c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 -- 172.21.15.32:0/3986695581 >> 172.21.15.32:0/3986695581 conn(0x7eff2c0fbe60 msgr2=0x7eff2c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 -- 172.21.15.32:0/3986695581 shutdown_connections 2023-08-11T15:44:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.447+0000 7eff33fff700 1 -- 172.21.15.32:0/3986695581 wait complete. 2023-08-11T15:44:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.448+0000 7eff33fff700 1 Processor -- start 2023-08-11T15:44:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff33fff700 1 -- start start 2023-08-11T15:44:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff33fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff2c100930 0x7eff2c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c101b20 0x7eff2c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff33fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff2c1024c0 0x7eff2c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff33fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1a0390 con 0x7eff2c101b20 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff33fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1a04d0 con 0x7eff2c1024c0 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff33fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1a0660 con 0x7eff2c100930 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff3159a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c101b20 0x7eff2c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff3159a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c101b20 0x7eff2c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47098/0 (socket says 172.21.15.32:47098) 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff31d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff2c100930 0x7eff2c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff3159a700 1 -- 172.21.15.32:0/996023411 learned_addr learned my addr 172.21.15.32:0/996023411 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.449+0000 7eff3259c700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff2c1024c0 0x7eff2c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff3159a700 1 -- 172.21.15.32:0/996023411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff2c100930 msgr2=0x7eff2c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff3159a700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff2c100930 0x7eff2c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff3159a700 1 -- 172.21.15.32:0/996023411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff2c1024c0 msgr2=0x7eff2c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff3159a700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff2c1024c0 0x7eff2c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff3159a700 1 -- 172.21.15.32:0/996023411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280095f0 con 0x7eff2c101b20 2023-08-11T15:44:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff3159a700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c101b20 0x7eff2c198d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00e9a0 tx=0x7eff1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff22ffd700 1 -- 172.21.15.32:0/996023411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c009790 con 0x7eff2c101b20 2023-08-11T15:44:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff22ffd700 1 -- 172.21.15.32:0/996023411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1c010430 con 0x7eff2c101b20 2023-08-11T15:44:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c1a0940 con 0x7eff2c101b20 2023-08-11T15:44:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff22ffd700 1 -- 172.21.15.32:0/996023411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c018680 con 0x7eff2c101b20 2023-08-11T15:44:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.450+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1a0e50 con 0x7eff2c101b20 2023-08-11T15:44:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.452+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c10c190 con 0x7eff2c101b20 2023-08-11T15:44:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.453+0000 7eff22ffd700 1 -- 172.21.15.32:0/996023411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7eff1c0105a0 con 0x7eff2c101b20 2023-08-11T15:44:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.456+0000 7eff22ffd700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff180a0020 0x7eff180a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.456+0000 7eff22ffd700 1 -- 172.21.15.32:0/996023411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7eff1c014070 con 0x7eff2c101b20 2023-08-11T15:44:56.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.456+0000 7eff31d9b700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff180a0020 0x7eff180a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.456+0000 7eff31d9b700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff180a0020 0x7eff180a24e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7eff28009990 tx=0x7eff28005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.460+0000 7eff22ffd700 1 -- 172.21.15.32:0/996023411 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7eff1c051c30 con 0x7eff2c101b20 2023-08-11T15:44:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:56 smithi134 ceph-mon[130447]: osdmap e90: 12 total, 12 up, 12 in 2023-08-11T15:44:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:56 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3078516862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:56 smithi032 ceph-mon[97103]: osdmap e90: 12 total, 12 up, 12 in 2023-08-11T15:44:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:56 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3078516862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:56.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:56 smithi192 ceph-mon[130806]: osdmap e90: 12 total, 12 up, 12 in 2023-08-11T15:44:56.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:56 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3078516862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:44:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.667+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff2c108440 con 0x7eff180a0020 2023-08-11T15:44:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.679+0000 7eff22ffd700 1 -- 172.21.15.32:0/996023411 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422261 (secure 0 0 0) 0x7eff2c108440 con 0x7eff180a0020 2023-08-11T15:44:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff180a0020 msgr2=0x7eff180a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff180a0020 0x7eff180a24e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7eff28009990 tx=0x7eff28005fb0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c101b20 msgr2=0x7eff2c198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c101b20 0x7eff2c198d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00e9a0 tx=0x7eff1c00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 shutdown_connections 2023-08-11T15:44:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7eff180a0020 0x7eff180a24e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff2c1024c0 0x7eff2c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff2c100930 0x7eff2c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 --2- 172.21.15.32:0/996023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c101b20 0x7eff2c198d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 >> 172.21.15.32:0/996023411 conn(0x7eff2c0fbe60 msgr2=0x7eff2c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 shutdown_connections 2023-08-11T15:44:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.685+0000 7eff33fff700 1 -- 172.21.15.32:0/996023411 wait complete. 2023-08-11T15:44:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-08-11T15:44:56.706 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-08-11T15:44:56.075793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595131,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":147,"num_read_kb":131,"num_write":231,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337400,"kb_used_data":13148,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604424,"statfs":{"total":1151940427776,"available":1151594930176,"internally_reserved":0,"allocated":13463552,"data_stored":7420815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002474"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184488+0000","last_change":"2023-08-11T15:44:19.862927+0000","last_active":"2023-08-11T15:44:25.184488+0000","last_peered":"2023-08-11T15:44:25.184488+0000","last_clean":"2023-08-11T15:44:25.184488+0000","last_became_active":"2023-08-11T15:44:19.862626+0000","last_became_peered":"2023-08-11T15:44:19.862626+0000","last_unstale":"2023-08-11T15:44:25.184488+0000","last_undegraded":"2023-08-11T15:44:25.184488+0000","last_fullsized":"2023-08-11T15:44:25.184488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:58:44.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241036+0000","last_change":"2023-08-11T15:44:19.852455+0000","last_active":"2023-08-11T15:44:26.241036+0000","last_peered":"2023-08-11T15:44:26.241036+0000","last_clean":"2023-08-11T15:44:26.241036+0000","last_became_active":"2023-08-11T15:44:19.851897+0000","last_became_peered":"2023-08-11T15:44:19.851897+0000","last_unstale":"2023-08-11T15:44:26.241036+0000","last_undegraded":"2023-08-11T15:44:26.241036+0000","last_fullsized":"2023-08-11T15:44:26.241036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:20.816842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241013+0000","last_change":"2023-08-11T15:44:19.855232+0000","last_active":"2023-08-11T15:44:26.241013+0000","last_peered":"2023-08-11T15:44:26.241013+0000","last_clean":"2023-08-11T15:44:26.241013+0000","last_became_active":"2023-08-11T15:44:19.854940+0000","last_became_peered":"2023-08-11T15:44:19.854940+0000","last_unstale":"2023-08-11T15:44:26.241013+0000","last_undegraded":"2023-08-11T15:44:26.241013+0000","last_fullsized":"2023-08-11T15:44:26.241013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:42:52.881533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196583+0000","last_change":"2023-08-11T15:44:19.856137+0000","last_active":"2023-08-11T15:44:25.196583+0000","last_peered":"2023-08-11T15:44:25.196583+0000","last_clean":"2023-08-11T15:44:25.196583+0000","last_became_active":"2023-08-11T15:44:19.855852+0000","last_became_peered":"2023-08-11T15:44:19.855852+0000","last_unstale":"2023-08-11T15:44:25.196583+0000","last_undegraded":"2023-08-11T15:44:25.196583+0000","last_fullsized":"2023-08-11T15:44:25.196583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:21.362833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194351+0000","last_change":"2023-08-11T15:44:19.862618+0000","last_active":"2023-08-11T15:44:54.194351+0000","last_peered":"2023-08-11T15:44:54.194351+0000","last_clean":"2023-08-11T15:44:54.194351+0000","last_became_active":"2023-08-11T15:44:19.862265+0000","last_became_peered":"2023-08-11T15:44:19.862265+0000","last_unstale":"2023-08-11T15:44:54.194351+0000","last_undegraded":"2023-08-11T15:44:54.194351+0000","last_fullsized":"2023-08-11T15:44:54.194351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:53.783047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909293+0000","last_change":"2023-08-11T15:44:19.857801+0000","last_active":"2023-08-11T15:44:26.909293+0000","last_peered":"2023-08-11T15:44:26.909293+0000","last_clean":"2023-08-11T15:44:26.909293+0000","last_became_active":"2023-08-11T15:44:19.857388+0000","last_became_peered":"2023-08-11T15:44:19.857388+0000","last_unstale":"2023-08-11T15:44:26.909293+0000","last_undegraded":"2023-08-11T15:44:26.909293+0000","last_fullsized":"2023-08-11T15:44:26.909293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:55:45.696116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189336+0000","last_change":"2023-08-11T15:44:19.859334+0000","last_active":"2023-08-11T15:44:54.189336+0000","last_peered":"2023-08-11T15:44:54.189336+0000","last_clean":"2023-08-11T15:44:54.189336+0000","last_became_active":"2023-08-11T15:44:19.859073+0000","last_became_peered":"2023-08-11T15:44:19.859073+0000","last_unstale":"2023-08-11T15:44:54.189336+0000","last_undegraded":"2023-08-11T15:44:54.189336+0000","last_fullsized":"2023-08-11T15:44:54.189336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:37:20.633727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178676+0000","last_change":"2023-08-11T15:44:19.853064+0000","last_active":"2023-08-11T15:44:25.178676+0000","last_peered":"2023-08-11T15:44:25.178676+0000","last_clean":"2023-08-11T15:44:25.178676+0000","last_became_active":"2023-08-11T15:44:19.852736+0000","last_became_peered":"2023-08-11T15:44:19.852736+0000","last_unstale":"2023-08-11T15:44:25.178676+0000","last_undegraded":"2023-08-11T15:44:25.178676+0000","last_fullsized":"2023-08-11T15:44:25.178676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:31.513054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497471+0000","last_change":"2023-08-11T15:44:19.861912+0000","last_active":"2023-08-11T15:44:27.497471+0000","last_peered":"2023-08-11T15:44:27.497471+0000","last_clean":"2023-08-11T15:44:27.497471+0000","last_became_active":"2023-08-11T15:44:19.861374+0000","last_became_peered":"2023-08-11T15:44:19.861374+0000","last_unstale":"2023-08-11T15:44:27.497471+0000","last_undegraded":"2023-08-11T15:44:27.497471+0000","last_fullsized":"2023-08-11T15:44:27.497471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:55.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909786+0000","last_change":"2023-08-11T15:44:19.858862+0000","last_active":"2023-08-11T15:44:26.909786+0000","last_peered":"2023-08-11T15:44:26.909786+0000","last_clean":"2023-08-11T15:44:26.909786+0000","last_became_active":"2023-08-11T15:44:19.858512+0000","last_became_peered":"2023-08-11T15:44:19.858512+0000","last_unstale":"2023-08-11T15:44:26.909786+0000","last_undegraded":"2023-08-11T15:44:26.909786+0000","last_fullsized":"2023-08-11T15:44:26.909786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:53:27.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190057+0000","last_change":"2023-08-11T15:44:19.864063+0000","last_active":"2023-08-11T15:44:54.190057+0000","last_peered":"2023-08-11T15:44:54.190057+0000","last_clean":"2023-08-11T15:44:54.190057+0000","last_became_active":"2023-08-11T15:44:19.863081+0000","last_became_peered":"2023-08-11T15:44:19.863081+0000","last_unstale":"2023-08-11T15:44:54.190057+0000","last_undegraded":"2023-08-11T15:44:54.190057+0000","last_fullsized":"2023-08-11T15:44:54.190057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:19:47.689166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196608+0000","last_change":"2023-08-11T15:44:19.855347+0000","last_active":"2023-08-11T15:44:25.196608+0000","last_peered":"2023-08-11T15:44:25.196608+0000","last_clean":"2023-08-11T15:44:25.196608+0000","last_became_active":"2023-08-11T15:44:19.855025+0000","last_became_peered":"2023-08-11T15:44:19.855025+0000","last_unstale":"2023-08-11T15:44:25.196608+0000","last_undegraded":"2023-08-11T15:44:25.196608+0000","last_fullsized":"2023-08-11T15:44:25.196608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:52:42.424929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398761+0000","last_change":"2023-08-11T15:44:19.862153+0000","last_active":"2023-08-11T15:44:25.398761+0000","last_peered":"2023-08-11T15:44:25.398761+0000","last_clean":"2023-08-11T15:44:25.398761+0000","last_became_active":"2023-08-11T15:44:19.861830+0000","last_became_peered":"2023-08-11T15:44:19.861830+0000","last_unstale":"2023-08-11T15:44:25.398761+0000","last_undegraded":"2023-08-11T15:44:25.398761+0000","last_fullsized":"2023-08-11T15:44:25.398761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:54:03.374483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240557+0000","last_change":"2023-08-11T15:44:19.857184+0000","last_active":"2023-08-11T15:44:26.240557+0000","last_peered":"2023-08-11T15:44:26.240557+0000","last_clean":"2023-08-11T15:44:26.240557+0000","last_became_active":"2023-08-11T15:44:19.856898+0000","last_became_peered":"2023-08-11T15:44:19.856898+0000","last_unstale":"2023-08-11T15:44:26.240557+0000","last_undegraded":"2023-08-11T15:44:26.240557+0000","last_fullsized":"2023-08-11T15:44:26.240557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:16.949556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240612+0000","last_change":"2023-08-11T15:44:19.852808+0000","last_active":"2023-08-11T15:44:26.240612+0000","last_peered":"2023-08-11T15:44:26.240612+0000","last_clean":"2023-08-11T15:44:26.240612+0000","last_became_active":"2023-08-11T15:44:19.852477+0000","last_became_peered":"2023-08-11T15:44:19.852477+0000","last_unstale":"2023-08-11T15:44:26.240612+0000","last_undegraded":"2023-08-11T15:44:26.240612+0000","last_fullsized":"2023-08-11T15:44:26.240612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:44:30.565373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496877+0000","last_change":"2023-08-11T15:44:19.856273+0000","last_active":"2023-08-11T15:44:27.496877+0000","last_peered":"2023-08-11T15:44:27.496877+0000","last_clean":"2023-08-11T15:44:27.496877+0000","last_became_active":"2023-08-11T15:44:19.854579+0000","last_became_peered":"2023-08-11T15:44:19.854579+0000","last_unstale":"2023-08-11T15:44:27.496877+0000","last_undegraded":"2023-08-11T15:44:27.496877+0000","last_fullsized":"2023-08-11T15:44:27.496877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:49:32.949731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184320+0000","last_change":"2023-08-11T15:44:19.866820+0000","last_active":"2023-08-11T15:44:25.184320+0000","last_peered":"2023-08-11T15:44:25.184320+0000","last_clean":"2023-08-11T15:44:25.184320+0000","last_became_active":"2023-08-11T15:44:19.866507+0000","last_became_peered":"2023-08-11T15:44:19.866507+0000","last_unstale":"2023-08-11T15:44:25.184320+0000","last_undegraded":"2023-08-11T15:44:25.184320+0000","last_fullsized":"2023-08-11T15:44:25.184320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:46.966328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181871+0000","last_change":"2023-08-11T15:44:19.865821+0000","last_active":"2023-08-11T15:44:25.181871+0000","last_peered":"2023-08-11T15:44:25.181871+0000","last_clean":"2023-08-11T15:44:25.181871+0000","last_became_active":"2023-08-11T15:44:19.865510+0000","last_became_peered":"2023-08-11T15:44:19.865510+0000","last_unstale":"2023-08-11T15:44:25.181871+0000","last_undegraded":"2023-08-11T15:44:25.181871+0000","last_fullsized":"2023-08-11T15:44:25.181871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:25:42.382893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130810+0000","last_change":"2023-08-11T15:44:19.863079+0000","last_active":"2023-08-11T15:44:27.130810+0000","last_peered":"2023-08-11T15:44:27.130810+0000","last_clean":"2023-08-11T15:44:27.130810+0000","last_became_active":"2023-08-11T15:44:19.862663+0000","last_became_peered":"2023-08-11T15:44:19.862663+0000","last_unstale":"2023-08-11T15:44:27.130810+0000","last_undegraded":"2023-08-11T15:44:27.130810+0000","last_fullsized":"2023-08-11T15:44:27.130810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:53.635387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130328+0000","last_change":"2023-08-11T15:44:24.201308+0000","last_active":"2023-08-11T15:44:27.130328+0000","last_peered":"2023-08-11T15:44:27.130328+0000","last_clean":"2023-08-11T15:44:27.130328+0000","last_became_active":"2023-08-11T15:44:24.200884+0000","last_became_peered":"2023-08-11T15:44:24.200884+0000","last_unstale":"2023-08-11T15:44:27.130328+0000","last_undegraded":"2023-08-11T15:44:27.130328+0000","last_fullsized":"2023-08-11T15:44:27.130328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:04:43.367558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181892+0000","last_change":"2023-08-11T15:44:19.155621+0000","last_active":"2023-08-11T15:44:25.181892+0000","last_peered":"2023-08-11T15:44:25.181892+0000","last_clean":"2023-08-11T15:44:25.181892+0000","last_became_active":"2023-08-11T15:44:19.154494+0000","last_became_peered":"2023-08-11T15:44:19.154494+0000","last_unstale":"2023-08-11T15:44:25.181892+0000","last_undegraded":"2023-08-11T15:44:25.181892+0000","last_fullsized":"2023-08-11T15:44:25.181892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:22:15.734115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179239+0000","last_change":"2023-08-11T15:44:19.853101+0000","last_active":"2023-08-11T15:44:25.179239+0000","last_peered":"2023-08-11T15:44:25.179239+0000","last_clean":"2023-08-11T15:44:25.179239+0000","last_became_active":"2023-08-11T15:44:19.852804+0000","last_became_peered":"2023-08-11T15:44:19.852804+0000","last_unstale":"2023-08-11T15:44:25.179239+0000","last_undegraded":"2023-08-11T15:44:25.179239+0000","last_fullsized":"2023-08-11T15:44:25.179239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:04:34.124031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196052+0000","last_change":"2023-08-11T15:44:24.198154+0000","last_active":"2023-08-11T15:44:25.196052+0000","last_peered":"2023-08-11T15:44:25.196052+0000","last_clean":"2023-08-11T15:44:25.196052+0000","last_became_active":"2023-08-11T15:44:24.197693+0000","last_became_peered":"2023-08-11T15:44:24.197693+0000","last_unstale":"2023-08-11T15:44:25.196052+0000","last_undegraded":"2023-08-11T15:44:25.196052+0000","last_fullsized":"2023-08-11T15:44:25.196052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:41:28.478746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179250+0000","last_change":"2023-08-11T15:44:19.147219+0000","last_active":"2023-08-11T15:44:25.179250+0000","last_peered":"2023-08-11T15:44:25.179250+0000","last_clean":"2023-08-11T15:44:25.179250+0000","last_became_active":"2023-08-11T15:44:19.146874+0000","last_became_peered":"2023-08-11T15:44:19.146874+0000","last_unstale":"2023-08-11T15:44:25.179250+0000","last_undegraded":"2023-08-11T15:44:25.179250+0000","last_fullsized":"2023-08-11T15:44:25.179250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:22:17.797682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241145+0000","last_change":"2023-08-11T15:44:19.854103+0000","last_active":"2023-08-11T15:44:26.241145+0000","last_peered":"2023-08-11T15:44:26.241145+0000","last_clean":"2023-08-11T15:44:26.241145+0000","last_became_active":"2023-08-11T15:44:19.853542+0000","last_became_peered":"2023-08-11T15:44:19.853542+0000","last_unstale":"2023-08-11T15:44:26.241145+0000","last_undegraded":"2023-08-11T15:44:26.241145+0000","last_fullsized":"2023-08-11T15:44:26.241145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:55:02.787594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196286+0000","last_change":"2023-08-11T15:44:24.202825+0000","last_active":"2023-08-11T15:44:25.196286+0000","last_peered":"2023-08-11T15:44:25.196286+0000","last_clean":"2023-08-11T15:44:25.196286+0000","last_became_active":"2023-08-11T15:44:24.202342+0000","last_became_peered":"2023-08-11T15:44:24.202342+0000","last_unstale":"2023-08-11T15:44:25.196286+0000","last_undegraded":"2023-08-11T15:44:25.196286+0000","last_fullsized":"2023-08-11T15:44:25.196286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:00.909726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196861+0000","last_change":"2023-08-11T15:44:19.148019+0000","last_active":"2023-08-11T15:44:54.196861+0000","last_peered":"2023-08-11T15:44:54.196861+0000","last_clean":"2023-08-11T15:44:54.196861+0000","last_became_active":"2023-08-11T15:44:19.147501+0000","last_became_peered":"2023-08-11T15:44:19.147501+0000","last_unstale":"2023-08-11T15:44:54.196861+0000","last_undegraded":"2023-08-11T15:44:54.196861+0000","last_fullsized":"2023-08-11T15:44:54.196861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:35.232097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497109+0000","last_change":"2023-08-11T15:44:19.865230+0000","last_active":"2023-08-11T15:44:27.497109+0000","last_peered":"2023-08-11T15:44:27.497109+0000","last_clean":"2023-08-11T15:44:27.497109+0000","last_became_active":"2023-08-11T15:44:19.864838+0000","last_became_peered":"2023-08-11T15:44:19.864838+0000","last_unstale":"2023-08-11T15:44:27.497109+0000","last_undegraded":"2023-08-11T15:44:27.497109+0000","last_fullsized":"2023-08-11T15:44:27.497109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:43:54.021675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196125+0000","last_change":"2023-08-11T15:44:24.192282+0000","last_active":"2023-08-11T15:44:25.196125+0000","last_peered":"2023-08-11T15:44:25.196125+0000","last_clean":"2023-08-11T15:44:25.196125+0000","last_became_active":"2023-08-11T15:44:24.191972+0000","last_became_peered":"2023-08-11T15:44:24.191972+0000","last_unstale":"2023-08-11T15:44:25.196125+0000","last_undegraded":"2023-08-11T15:44:25.196125+0000","last_fullsized":"2023-08-11T15:44:25.196125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:39.427988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196635+0000","last_change":"2023-08-11T15:44:19.147181+0000","last_active":"2023-08-11T15:44:54.196635+0000","last_peered":"2023-08-11T15:44:54.196635+0000","last_clean":"2023-08-11T15:44:54.196635+0000","last_became_active":"2023-08-11T15:44:19.146843+0000","last_became_peered":"2023-08-11T15:44:19.146843+0000","last_unstale":"2023-08-11T15:44:54.196635+0000","last_undegraded":"2023-08-11T15:44:54.196635+0000","last_fullsized":"2023-08-11T15:44:54.196635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:36:39.248789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043920+0000","last_change":"2023-08-11T15:44:19.858611+0000","last_active":"2023-08-11T15:44:27.043920+0000","last_peered":"2023-08-11T15:44:27.043920+0000","last_clean":"2023-08-11T15:44:27.043920+0000","last_became_active":"2023-08-11T15:44:19.858288+0000","last_became_peered":"2023-08-11T15:44:19.858288+0000","last_unstale":"2023-08-11T15:44:27.043920+0000","last_undegraded":"2023-08-11T15:44:27.043920+0000","last_fullsized":"2023-08-11T15:44:27.043920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:41.338062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130820+0000","last_change":"2023-08-11T15:44:24.192621+0000","last_active":"2023-08-11T15:44:27.130820+0000","last_peered":"2023-08-11T15:44:27.130820+0000","last_clean":"2023-08-11T15:44:27.130820+0000","last_became_active":"2023-08-11T15:44:24.192183+0000","last_became_peered":"2023-08-11T15:44:24.192183+0000","last_unstale":"2023-08-11T15:44:27.130820+0000","last_undegraded":"2023-08-11T15:44:27.130820+0000","last_fullsized":"2023-08-11T15:44:27.130820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:01.082842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043685+0000","last_change":"2023-08-11T15:44:19.149437+0000","last_active":"2023-08-11T15:44:27.043685+0000","last_peered":"2023-08-11T15:44:27.043685+0000","last_clean":"2023-08-11T15:44:27.043685+0000","last_became_active":"2023-08-11T15:44:19.149086+0000","last_became_peered":"2023-08-11T15:44:19.149086+0000","last_unstale":"2023-08-11T15:44:27.043685+0000","last_undegraded":"2023-08-11T15:44:27.043685+0000","last_fullsized":"2023-08-11T15:44:27.043685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:38.263428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196160+0000","last_change":"2023-08-11T15:44:19.865452+0000","last_active":"2023-08-11T15:44:54.196160+0000","last_peered":"2023-08-11T15:44:54.196160+0000","last_clean":"2023-08-11T15:44:54.196160+0000","last_became_active":"2023-08-11T15:44:19.865133+0000","last_became_peered":"2023-08-11T15:44:19.865133+0000","last_unstale":"2023-08-11T15:44:54.196160+0000","last_undegraded":"2023-08-11T15:44:54.196160+0000","last_fullsized":"2023-08-11T15:44:54.196160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:20.210619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130658+0000","last_change":"2023-08-11T15:44:24.201817+0000","last_active":"2023-08-11T15:44:27.130658+0000","last_peered":"2023-08-11T15:44:27.130658+0000","last_clean":"2023-08-11T15:44:27.130658+0000","last_became_active":"2023-08-11T15:44:24.200766+0000","last_became_peered":"2023-08-11T15:44:24.200766+0000","last_unstale":"2023-08-11T15:44:27.130658+0000","last_undegraded":"2023-08-11T15:44:27.130658+0000","last_fullsized":"2023-08-11T15:44:27.130658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:44:12.538791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909254+0000","last_change":"2023-08-11T15:44:19.151213+0000","last_active":"2023-08-11T15:44:26.909254+0000","last_peered":"2023-08-11T15:44:26.909254+0000","last_clean":"2023-08-11T15:44:26.909254+0000","last_became_active":"2023-08-11T15:44:19.150914+0000","last_became_peered":"2023-08-11T15:44:19.150914+0000","last_unstale":"2023-08-11T15:44:26.909254+0000","last_undegraded":"2023-08-11T15:44:26.909254+0000","last_fullsized":"2023-08-11T15:44:26.909254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:33:38.997413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043814+0000","last_change":"2023-08-11T15:44:19.858238+0000","last_active":"2023-08-11T15:44:27.043814+0000","last_peered":"2023-08-11T15:44:27.043814+0000","last_clean":"2023-08-11T15:44:27.043814+0000","last_became_active":"2023-08-11T15:44:19.857919+0000","last_became_peered":"2023-08-11T15:44:19.857919+0000","last_unstale":"2023-08-11T15:44:27.043814+0000","last_undegraded":"2023-08-11T15:44:27.043814+0000","last_fullsized":"2023-08-11T15:44:27.043814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:20:28.876449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184406+0000","last_change":"2023-08-11T15:44:24.201485+0000","last_active":"2023-08-11T15:44:25.184406+0000","last_peered":"2023-08-11T15:44:25.184406+0000","last_clean":"2023-08-11T15:44:25.184406+0000","last_became_active":"2023-08-11T15:44:24.200984+0000","last_became_peered":"2023-08-11T15:44:24.200984+0000","last_unstale":"2023-08-11T15:44:25.184406+0000","last_undegraded":"2023-08-11T15:44:25.184406+0000","last_fullsized":"2023-08-11T15:44:25.184406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:12.802792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398401+0000","last_change":"2023-08-11T15:44:19.149289+0000","last_active":"2023-08-11T15:44:25.398401+0000","last_peered":"2023-08-11T15:44:25.398401+0000","last_clean":"2023-08-11T15:44:25.398401+0000","last_became_active":"2023-08-11T15:44:19.148881+0000","last_became_peered":"2023-08-11T15:44:19.148881+0000","last_unstale":"2023-08-11T15:44:25.398401+0000","last_undegraded":"2023-08-11T15:44:25.398401+0000","last_fullsized":"2023-08-11T15:44:25.398401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:31.674718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178473+0000","last_change":"2023-08-11T15:44:19.867860+0000","last_active":"2023-08-11T15:44:25.178473+0000","last_peered":"2023-08-11T15:44:25.178473+0000","last_clean":"2023-08-11T15:44:25.178473+0000","last_became_active":"2023-08-11T15:44:19.867545+0000","last_became_peered":"2023-08-11T15:44:19.867545+0000","last_unstale":"2023-08-11T15:44:25.178473+0000","last_undegraded":"2023-08-11T15:44:25.178473+0000","last_fullsized":"2023-08-11T15:44:25.178473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:22.607885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241050+0000","last_change":"2023-08-11T15:44:24.197251+0000","last_active":"2023-08-11T15:44:26.241050+0000","last_peered":"2023-08-11T15:44:26.241050+0000","last_clean":"2023-08-11T15:44:26.241050+0000","last_became_active":"2023-08-11T15:44:24.196735+0000","last_became_peered":"2023-08-11T15:44:24.196735+0000","last_unstale":"2023-08-11T15:44:26.241050+0000","last_undegraded":"2023-08-11T15:44:26.241050+0000","last_fullsized":"2023-08-11T15:44:26.241050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:28:12.008317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043322+0000","last_change":"2023-08-11T15:44:19.146196+0000","last_active":"2023-08-11T15:44:27.043322+0000","last_peered":"2023-08-11T15:44:27.043322+0000","last_clean":"2023-08-11T15:44:27.043322+0000","last_became_active":"2023-08-11T15:44:19.145812+0000","last_became_peered":"2023-08-11T15:44:19.145812+0000","last_unstale":"2023-08-11T15:44:27.043322+0000","last_undegraded":"2023-08-11T15:44:27.043322+0000","last_fullsized":"2023-08-11T15:44:27.043322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:57.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181289+0000","last_change":"2023-08-11T15:44:19.855389+0000","last_active":"2023-08-11T15:44:25.181289+0000","last_peered":"2023-08-11T15:44:25.181289+0000","last_clean":"2023-08-11T15:44:25.181289+0000","last_became_active":"2023-08-11T15:44:19.855063+0000","last_became_peered":"2023-08-11T15:44:19.855063+0000","last_unstale":"2023-08-11T15:44:25.181289+0000","last_undegraded":"2023-08-11T15:44:25.181289+0000","last_fullsized":"2023-08-11T15:44:25.181289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:06:46.738170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178587+0000","last_change":"2023-08-11T15:44:24.192450+0000","last_active":"2023-08-11T15:44:25.178587+0000","last_peered":"2023-08-11T15:44:25.178587+0000","last_clean":"2023-08-11T15:44:25.178587+0000","last_became_active":"2023-08-11T15:44:24.191989+0000","last_became_peered":"2023-08-11T15:44:24.191989+0000","last_unstale":"2023-08-11T15:44:25.178587+0000","last_undegraded":"2023-08-11T15:44:25.178587+0000","last_fullsized":"2023-08-11T15:44:25.178587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:54:21.852954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909733+0000","last_change":"2023-08-11T15:44:19.149002+0000","last_active":"2023-08-11T15:44:26.909733+0000","last_peered":"2023-08-11T15:44:26.909733+0000","last_clean":"2023-08-11T15:44:26.909733+0000","last_became_active":"2023-08-11T15:44:19.148635+0000","last_became_peered":"2023-08-11T15:44:19.148635+0000","last_unstale":"2023-08-11T15:44:26.909733+0000","last_undegraded":"2023-08-11T15:44:26.909733+0000","last_fullsized":"2023-08-11T15:44:26.909733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:02.711671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497221+0000","last_change":"2023-08-11T15:44:19.858067+0000","last_active":"2023-08-11T15:44:27.497221+0000","last_peered":"2023-08-11T15:44:27.497221+0000","last_clean":"2023-08-11T15:44:27.497221+0000","last_became_active":"2023-08-11T15:44:19.856641+0000","last_became_peered":"2023-08-11T15:44:19.856641+0000","last_unstale":"2023-08-11T15:44:27.497221+0000","last_undegraded":"2023-08-11T15:44:27.497221+0000","last_fullsized":"2023-08-11T15:44:27.497221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:29.896919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181349+0000","last_change":"2023-08-11T15:44:24.196806+0000","last_active":"2023-08-11T15:44:25.181349+0000","last_peered":"2023-08-11T15:44:25.181349+0000","last_clean":"2023-08-11T15:44:25.181349+0000","last_became_active":"2023-08-11T15:44:24.195268+0000","last_became_peered":"2023-08-11T15:44:24.195268+0000","last_unstale":"2023-08-11T15:44:25.181349+0000","last_undegraded":"2023-08-11T15:44:25.181349+0000","last_fullsized":"2023-08-11T15:44:25.181349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:45:38.922930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.496972+0000","last_change":"2023-08-11T15:44:54.190469+0000","last_active":"2023-08-11T15:44:54.190469+0000","last_peered":"2023-08-11T15:44:27.496972+0000","last_clean":"2023-08-11T15:44:27.496972+0000","last_became_active":"2023-08-11T15:44:19.144155+0000","last_became_peered":"2023-08-11T15:44:19.144155+0000","last_unstale":"2023-08-11T15:44:27.496972+0000","last_undegraded":"2023-08-11T15:44:27.496972+0000","last_fullsized":"2023-08-11T15:44:27.496972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240742+0000","last_change":"2023-08-11T15:44:19.853881+0000","last_active":"2023-08-11T15:44:26.240742+0000","last_peered":"2023-08-11T15:44:26.240742+0000","last_clean":"2023-08-11T15:44:26.240742+0000","last_became_active":"2023-08-11T15:44:19.853139+0000","last_became_peered":"2023-08-11T15:44:19.853139+0000","last_unstale":"2023-08-11T15:44:26.240742+0000","last_undegraded":"2023-08-11T15:44:26.240742+0000","last_fullsized":"2023-08-11T15:44:26.240742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:05.315621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196399+0000","last_change":"2023-08-11T15:44:24.201620+0000","last_active":"2023-08-11T15:44:25.196399+0000","last_peered":"2023-08-11T15:44:25.196399+0000","last_clean":"2023-08-11T15:44:25.196399+0000","last_became_active":"2023-08-11T15:44:24.201162+0000","last_became_peered":"2023-08-11T15:44:24.201162+0000","last_unstale":"2023-08-11T15:44:25.196399+0000","last_undegraded":"2023-08-11T15:44:25.196399+0000","last_fullsized":"2023-08-11T15:44:25.196399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:54:06.512139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043904+0000","last_change":"2023-08-11T15:44:19.156899+0000","last_active":"2023-08-11T15:44:27.043904+0000","last_peered":"2023-08-11T15:44:27.043904+0000","last_clean":"2023-08-11T15:44:27.043904+0000","last_became_active":"2023-08-11T15:44:19.156643+0000","last_became_peered":"2023-08-11T15:44:19.156643+0000","last_unstale":"2023-08-11T15:44:27.043904+0000","last_undegraded":"2023-08-11T15:44:27.043904+0000","last_fullsized":"2023-08-11T15:44:27.043904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:27.691696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497337+0000","last_change":"2023-08-11T15:44:19.857828+0000","last_active":"2023-08-11T15:44:27.497337+0000","last_peered":"2023-08-11T15:44:27.497337+0000","last_clean":"2023-08-11T15:44:27.497337+0000","last_became_active":"2023-08-11T15:44:19.856781+0000","last_became_peered":"2023-08-11T15:44:19.856781+0000","last_unstale":"2023-08-11T15:44:27.497337+0000","last_undegraded":"2023-08-11T15:44:27.497337+0000","last_fullsized":"2023-08-11T15:44:27.497337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:59.861717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181369+0000","last_change":"2023-08-11T15:44:24.193213+0000","last_active":"2023-08-11T15:44:25.181369+0000","last_peered":"2023-08-11T15:44:25.181369+0000","last_clean":"2023-08-11T15:44:25.181369+0000","last_became_active":"2023-08-11T15:44:24.192819+0000","last_became_peered":"2023-08-11T15:44:24.192819+0000","last_unstale":"2023-08-11T15:44:25.181369+0000","last_undegraded":"2023-08-11T15:44:25.181369+0000","last_fullsized":"2023-08-11T15:44:25.181369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:16.938740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'10","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:52.186980+0000","last_change":"2023-08-11T15:44:19.149753+0000","last_active":"2023-08-11T15:44:52.186980+0000","last_peered":"2023-08-11T15:44:52.186980+0000","last_clean":"2023-08-11T15:44:52.186980+0000","last_became_active":"2023-08-11T15:44:19.149420+0000","last_became_peered":"2023-08-11T15:44:19.149420+0000","last_unstale":"2023-08-11T15:44:52.186980+0000","last_undegraded":"2023-08-11T15:44:52.186980+0000","last_fullsized":"2023-08-11T15:44:52.186980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:58.376378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196006+0000","last_change":"2023-08-11T15:44:19.861342+0000","last_active":"2023-08-11T15:44:54.196006+0000","last_peered":"2023-08-11T15:44:54.196006+0000","last_clean":"2023-08-11T15:44:54.196006+0000","last_became_active":"2023-08-11T15:44:19.860978+0000","last_became_peered":"2023-08-11T15:44:19.860978+0000","last_unstale":"2023-08-11T15:44:54.196006+0000","last_undegraded":"2023-08-11T15:44:54.196006+0000","last_fullsized":"2023-08-11T15:44:54.196006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:52.637866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044274+0000","last_change":"2023-08-11T15:44:24.191504+0000","last_active":"2023-08-11T15:44:27.044274+0000","last_peered":"2023-08-11T15:44:27.044274+0000","last_clean":"2023-08-11T15:44:27.044274+0000","last_became_active":"2023-08-11T15:44:24.191127+0000","last_became_peered":"2023-08-11T15:44:24.191127+0000","last_unstale":"2023-08-11T15:44:27.044274+0000","last_undegraded":"2023-08-11T15:44:27.044274+0000","last_fullsized":"2023-08-11T15:44:27.044274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:11.970278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130897+0000","last_change":"2023-08-11T15:44:19.148354+0000","last_active":"2023-08-11T15:44:27.130897+0000","last_peered":"2023-08-11T15:44:27.130897+0000","last_clean":"2023-08-11T15:44:27.130897+0000","last_became_active":"2023-08-11T15:44:19.148036+0000","last_became_peered":"2023-08-11T15:44:19.148036+0000","last_unstale":"2023-08-11T15:44:27.130897+0000","last_undegraded":"2023-08-11T15:44:27.130897+0000","last_fullsized":"2023-08-11T15:44:27.130897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:33.516998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398521+0000","last_change":"2023-08-11T15:44:19.862409+0000","last_active":"2023-08-11T15:44:25.398521+0000","last_peered":"2023-08-11T15:44:25.398521+0000","last_clean":"2023-08-11T15:44:25.398521+0000","last_became_active":"2023-08-11T15:44:19.861359+0000","last_became_peered":"2023-08-11T15:44:19.861359+0000","last_unstale":"2023-08-11T15:44:25.398521+0000","last_undegraded":"2023-08-11T15:44:25.398521+0000","last_fullsized":"2023-08-11T15:44:25.398521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:41.755533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184424+0000","last_change":"2023-08-11T15:44:24.197634+0000","last_active":"2023-08-11T15:44:25.184424+0000","last_peered":"2023-08-11T15:44:25.184424+0000","last_clean":"2023-08-11T15:44:25.184424+0000","last_became_active":"2023-08-11T15:44:24.197181+0000","last_became_peered":"2023-08-11T15:44:24.197181+0000","last_unstale":"2023-08-11T15:44:25.184424+0000","last_undegraded":"2023-08-11T15:44:25.184424+0000","last_fullsized":"2023-08-11T15:44:25.184424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:23:45.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043675+0000","last_change":"2023-08-11T15:44:19.147550+0000","last_active":"2023-08-11T15:44:27.043675+0000","last_peered":"2023-08-11T15:44:27.043675+0000","last_clean":"2023-08-11T15:44:27.043675+0000","last_became_active":"2023-08-11T15:44:19.147299+0000","last_became_peered":"2023-08-11T15:44:19.147299+0000","last_unstale":"2023-08-11T15:44:27.043675+0000","last_undegraded":"2023-08-11T15:44:27.043675+0000","last_fullsized":"2023-08-11T15:44:27.043675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:56.050406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189968+0000","last_change":"2023-08-11T15:44:19.862664+0000","last_active":"2023-08-11T15:44:54.189968+0000","last_peered":"2023-08-11T15:44:54.189968+0000","last_clean":"2023-08-11T15:44:54.189968+0000","last_became_active":"2023-08-11T15:44:19.862130+0000","last_became_peered":"2023-08-11T15:44:19.862130+0000","last_unstale":"2023-08-11T15:44:54.189968+0000","last_undegraded":"2023-08-11T15:44:54.189968+0000","last_fullsized":"2023-08-11T15:44:54.189968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:02.591318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181786+0000","last_change":"2023-08-11T15:44:24.199851+0000","last_active":"2023-08-11T15:44:25.181786+0000","last_peered":"2023-08-11T15:44:25.181786+0000","last_clean":"2023-08-11T15:44:25.181786+0000","last_became_active":"2023-08-11T15:44:24.199420+0000","last_became_peered":"2023-08-11T15:44:24.199420+0000","last_unstale":"2023-08-11T15:44:25.181786+0000","last_undegraded":"2023-08-11T15:44:25.181786+0000","last_fullsized":"2023-08-11T15:44:25.181786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:36:31.272229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196088+0000","last_change":"2023-08-11T15:44:19.145888+0000","last_active":"2023-08-11T15:44:25.196088+0000","last_peered":"2023-08-11T15:44:25.196088+0000","last_clean":"2023-08-11T15:44:25.196088+0000","last_became_active":"2023-08-11T15:44:19.145459+0000","last_became_peered":"2023-08-11T15:44:19.145459+0000","last_unstale":"2023-08-11T15:44:25.196088+0000","last_undegraded":"2023-08-11T15:44:25.196088+0000","last_fullsized":"2023-08-11T15:44:25.196088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:36.387374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189722+0000","last_change":"2023-08-11T15:44:19.860035+0000","last_active":"2023-08-11T15:44:54.189722+0000","last_peered":"2023-08-11T15:44:54.189722+0000","last_clean":"2023-08-11T15:44:54.189722+0000","last_became_active":"2023-08-11T15:44:19.859696+0000","last_became_peered":"2023-08-11T15:44:19.859696+0000","last_unstale":"2023-08-11T15:44:54.189722+0000","last_undegraded":"2023-08-11T15:44:54.189722+0000","last_fullsized":"2023-08-11T15:44:54.189722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:24.878953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399086+0000","last_change":"2023-08-11T15:44:24.198255+0000","last_active":"2023-08-11T15:44:25.399086+0000","last_peered":"2023-08-11T15:44:25.399086+0000","last_clean":"2023-08-11T15:44:25.399086+0000","last_became_active":"2023-08-11T15:44:24.197825+0000","last_became_peered":"2023-08-11T15:44:24.197825+0000","last_unstale":"2023-08-11T15:44:25.399086+0000","last_undegraded":"2023-08-11T15:44:25.399086+0000","last_fullsized":"2023-08-11T15:44:25.399086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:41:49.976978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496775+0000","last_change":"2023-08-11T15:44:19.151410+0000","last_active":"2023-08-11T15:44:27.496775+0000","last_peered":"2023-08-11T15:44:27.496775+0000","last_clean":"2023-08-11T15:44:27.496775+0000","last_became_active":"2023-08-11T15:44:19.151117+0000","last_became_peered":"2023-08-11T15:44:19.151117+0000","last_unstale":"2023-08-11T15:44:27.496775+0000","last_undegraded":"2023-08-11T15:44:27.496775+0000","last_fullsized":"2023-08-11T15:44:27.496775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:11.127495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043467+0000","last_change":"2023-08-11T15:44:19.853273+0000","last_active":"2023-08-11T15:44:27.043467+0000","last_peered":"2023-08-11T15:44:27.043467+0000","last_clean":"2023-08-11T15:44:27.043467+0000","last_became_active":"2023-08-11T15:44:19.852914+0000","last_became_peered":"2023-08-11T15:44:19.852914+0000","last_unstale":"2023-08-11T15:44:27.043467+0000","last_undegraded":"2023-08-11T15:44:27.043467+0000","last_fullsized":"2023-08-11T15:44:27.043467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:45.110625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043436+0000","last_change":"2023-08-11T15:44:24.194429+0000","last_active":"2023-08-11T15:44:27.043436+0000","last_peered":"2023-08-11T15:44:27.043436+0000","last_clean":"2023-08-11T15:44:27.043436+0000","last_became_active":"2023-08-11T15:44:24.194089+0000","last_became_peered":"2023-08-11T15:44:24.194089+0000","last_unstale":"2023-08-11T15:44:27.043436+0000","last_undegraded":"2023-08-11T15:44:27.043436+0000","last_fullsized":"2023-08-11T15:44:27.043436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:53.530309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194680+0000","last_change":"2023-08-11T15:44:19.151072+0000","last_active":"2023-08-11T15:44:54.194680+0000","last_peered":"2023-08-11T15:44:54.194680+0000","last_clean":"2023-08-11T15:44:54.194680+0000","last_became_active":"2023-08-11T15:44:19.147713+0000","last_became_peered":"2023-08-11T15:44:19.147713+0000","last_unstale":"2023-08-11T15:44:54.194680+0000","last_undegraded":"2023-08-11T15:44:54.194680+0000","last_fullsized":"2023-08-11T15:44:54.194680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:09:42.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184182+0000","last_change":"2023-08-11T15:44:19.862259+0000","last_active":"2023-08-11T15:44:25.184182+0000","last_peered":"2023-08-11T15:44:25.184182+0000","last_clean":"2023-08-11T15:44:25.184182+0000","last_became_active":"2023-08-11T15:44:19.861910+0000","last_became_peered":"2023-08-11T15:44:19.861910+0000","last_unstale":"2023-08-11T15:44:25.184182+0000","last_undegraded":"2023-08-11T15:44:25.184182+0000","last_fullsized":"2023-08-11T15:44:25.184182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:06:41.610675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.497633+0000","last_change":"2023-08-11T15:44:54.195412+0000","last_active":"2023-08-11T15:44:54.195412+0000","last_peered":"2023-08-11T15:44:27.497633+0000","last_clean":"2023-08-11T15:44:27.497633+0000","last_became_active":"2023-08-11T15:44:24.300268+0000","last_became_peered":"2023-08-11T15:44:24.300268+0000","last_unstale":"2023-08-11T15:44:27.497633+0000","last_undegraded":"2023-08-11T15:44:27.497633+0000","last_fullsized":"2023-08-11T15:44:27.497633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181579+0000","last_change":"2023-08-11T15:44:19.152608+0000","last_active":"2023-08-11T15:44:25.181579+0000","last_peered":"2023-08-11T15:44:25.181579+0000","last_clean":"2023-08-11T15:44:25.181579+0000","last_became_active":"2023-08-11T15:44:19.152255+0000","last_became_peered":"2023-08-11T15:44:19.152255+0000","last_unstale":"2023-08-11T15:44:25.181579+0000","last_undegraded":"2023-08-11T15:44:25.181579+0000","last_fullsized":"2023-08-11T15:44:25.181579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:33.299760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909653+0000","last_change":"2023-08-11T15:44:19.864003+0000","last_active":"2023-08-11T15:44:26.909653+0000","last_peered":"2023-08-11T15:44:26.909653+0000","last_clean":"2023-08-11T15:44:26.909653+0000","last_became_active":"2023-08-11T15:44:19.863661+0000","last_became_peered":"2023-08-11T15:44:19.863661+0000","last_unstale":"2023-08-11T15:44:26.909653+0000","last_undegraded":"2023-08-11T15:44:26.909653+0000","last_fullsized":"2023-08-11T15:44:26.909653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:20:35.118164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398869+0000","last_change":"2023-08-11T15:44:24.194123+0000","last_active":"2023-08-11T15:44:25.398869+0000","last_peered":"2023-08-11T15:44:25.398869+0000","last_clean":"2023-08-11T15:44:25.398869+0000","last_became_active":"2023-08-11T15:44:24.193729+0000","last_became_peered":"2023-08-11T15:44:24.193729+0000","last_unstale":"2023-08-11T15:44:25.398869+0000","last_undegraded":"2023-08-11T15:44:25.398869+0000","last_fullsized":"2023-08-11T15:44:25.398869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:11:10.722179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189631+0000","last_change":"2023-08-11T15:44:19.151367+0000","last_active":"2023-08-11T15:44:54.189631+0000","last_peered":"2023-08-11T15:44:54.189631+0000","last_clean":"2023-08-11T15:44:54.189631+0000","last_became_active":"2023-08-11T15:44:19.150991+0000","last_became_peered":"2023-08-11T15:44:19.150991+0000","last_unstale":"2023-08-11T15:44:54.189631+0000","last_undegraded":"2023-08-11T15:44:54.189631+0000","last_fullsized":"2023-08-11T15:44:54.189631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:03:03.249036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909318+0000","last_change":"2023-08-11T15:44:19.858579+0000","last_active":"2023-08-11T15:44:26.909318+0000","last_peered":"2023-08-11T15:44:26.909318+0000","last_clean":"2023-08-11T15:44:26.909318+0000","last_became_active":"2023-08-11T15:44:19.858184+0000","last_became_peered":"2023-08-11T15:44:19.858184+0000","last_unstale":"2023-08-11T15:44:26.909318+0000","last_undegraded":"2023-08-11T15:44:26.909318+0000","last_fullsized":"2023-08-11T15:44:26.909318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:38.034222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399074+0000","last_change":"2023-08-11T15:44:24.300526+0000","last_active":"2023-08-11T15:44:25.399074+0000","last_peered":"2023-08-11T15:44:25.399074+0000","last_clean":"2023-08-11T15:44:25.399074+0000","last_became_active":"2023-08-11T15:44:24.299883+0000","last_became_peered":"2023-08-11T15:44:24.299883+0000","last_unstale":"2023-08-11T15:44:25.399074+0000","last_undegraded":"2023-08-11T15:44:25.399074+0000","last_fullsized":"2023-08-11T15:44:25.399074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:53:45.487528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043468+0000","last_change":"2023-08-11T15:44:19.858116+0000","last_active":"2023-08-11T15:44:27.043468+0000","last_peered":"2023-08-11T15:44:27.043468+0000","last_clean":"2023-08-11T15:44:27.043468+0000","last_became_active":"2023-08-11T15:44:19.857728+0000","last_became_peered":"2023-08-11T15:44:19.857728+0000","last_unstale":"2023-08-11T15:44:27.043468+0000","last_undegraded":"2023-08-11T15:44:27.043468+0000","last_fullsized":"2023-08-11T15:44:27.043468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:56:36.953948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398236+0000","last_change":"2023-08-11T15:44:19.157904+0000","last_active":"2023-08-11T15:44:25.398236+0000","last_peered":"2023-08-11T15:44:25.398236+0000","last_clean":"2023-08-11T15:44:25.398236+0000","last_became_active":"2023-08-11T15:44:19.157497+0000","last_became_peered":"2023-08-11T15:44:19.157497+0000","last_unstale":"2023-08-11T15:44:25.398236+0000","last_undegraded":"2023-08-11T15:44:25.398236+0000","last_fullsized":"2023-08-11T15:44:25.398236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:10.337630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189839+0000","last_change":"2023-08-11T15:44:24.201246+0000","last_active":"2023-08-11T15:44:54.189839+0000","last_peered":"2023-08-11T15:44:54.189839+0000","last_clean":"2023-08-11T15:44:54.189839+0000","last_became_active":"2023-08-11T15:44:24.200810+0000","last_became_peered":"2023-08-11T15:44:24.200810+0000","last_unstale":"2023-08-11T15:44:54.189839+0000","last_undegraded":"2023-08-11T15:44:54.189839+0000","last_fullsized":"2023-08-11T15:44:54.189839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:47:27.192243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178569+0000","last_change":"2023-08-11T15:44:19.846296+0000","last_active":"2023-08-11T15:44:25.178569+0000","last_peered":"2023-08-11T15:44:25.178569+0000","last_clean":"2023-08-11T15:44:25.178569+0000","last_became_active":"2023-08-11T15:44:19.845890+0000","last_became_peered":"2023-08-11T15:44:19.845890+0000","last_unstale":"2023-08-11T15:44:25.178569+0000","last_undegraded":"2023-08-11T15:44:25.178569+0000","last_fullsized":"2023-08-11T15:44:25.178569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:44:22.834705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196213+0000","last_change":"2023-08-11T15:44:19.144202+0000","last_active":"2023-08-11T15:44:25.196213+0000","last_peered":"2023-08-11T15:44:25.196213+0000","last_clean":"2023-08-11T15:44:25.196213+0000","last_became_active":"2023-08-11T15:44:19.143789+0000","last_became_peered":"2023-08-11T15:44:19.143789+0000","last_unstale":"2023-08-11T15:44:25.196213+0000","last_undegraded":"2023-08-11T15:44:25.196213+0000","last_fullsized":"2023-08-11T15:44:25.196213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:57:56.919463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189289+0000","last_change":"2023-08-11T15:43:14.593634+0000","last_active":"2023-08-11T15:44:54.189289+0000","last_peered":"2023-08-11T15:44:54.189289+0000","last_clean":"2023-08-11T15:44:54.189289+0000","last_became_active":"2023-08-11T15:43:14.593164+0000","last_became_peered":"2023-08-11T15:43:14.593164+0000","last_unstale":"2023-08-11T15:44:54.189289+0000","last_undegraded":"2023-08-11T15:44:54.189289+0000","last_fullsized":"2023-08-11T15:44:54.189289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_clean_scrub_stamp":"2023-08-11T15:40:09.009169+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:15.991425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044055+0000","last_change":"2023-08-11T15:44:24.300703+0000","last_active":"2023-08-11T15:44:27.044055+0000","last_peered":"2023-08-11T15:44:27.044055+0000","last_clean":"2023-08-11T15:44:27.044055+0000","last_became_active":"2023-08-11T15:44:24.300309+0000","last_became_peered":"2023-08-11T15:44:24.300309+0000","last_unstale":"2023-08-11T15:44:27.044055+0000","last_undegraded":"2023-08-11T15:44:27.044055+0000","last_fullsized":"2023-08-11T15:44:27.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:41.018379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398223+0000","last_change":"2023-08-11T15:44:19.858934+0000","last_active":"2023-08-11T15:44:25.398223+0000","last_peered":"2023-08-11T15:44:25.398223+0000","last_clean":"2023-08-11T15:44:25.398223+0000","last_became_active":"2023-08-11T15:44:19.858613+0000","last_became_peered":"2023-08-11T15:44:19.858613+0000","last_unstale":"2023-08-11T15:44:25.398223+0000","last_undegraded":"2023-08-11T15:44:25.398223+0000","last_fullsized":"2023-08-11T15:44:25.398223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:07.938228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189544+0000","last_change":"2023-08-11T15:44:19.148113+0000","last_active":"2023-08-11T15:44:54.189544+0000","last_peered":"2023-08-11T15:44:54.189544+0000","last_clean":"2023-08-11T15:44:54.189544+0000","last_became_active":"2023-08-11T15:44:19.147793+0000","last_became_peered":"2023-08-11T15:44:19.147793+0000","last_unstale":"2023-08-11T15:44:54.189544+0000","last_undegraded":"2023-08-11T15:44:54.189544+0000","last_fullsized":"2023-08-11T15:44:54.189544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:07:38.923167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184402+0000","last_change":"2023-08-11T15:44:24.194418+0000","last_active":"2023-08-11T15:44:25.184402+0000","last_peered":"2023-08-11T15:44:25.184402+0000","last_clean":"2023-08-11T15:44:25.184402+0000","last_became_active":"2023-08-11T15:44:24.193943+0000","last_became_peered":"2023-08-11T15:44:24.193943+0000","last_unstale":"2023-08-11T15:44:25.184402+0000","last_undegraded":"2023-08-11T15:44:25.184402+0000","last_fullsized":"2023-08-11T15:44:25.184402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:35.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196145+0000","last_change":"2023-08-11T15:44:19.151476+0000","last_active":"2023-08-11T15:44:54.196145+0000","last_peered":"2023-08-11T15:44:54.196145+0000","last_clean":"2023-08-11T15:44:54.196145+0000","last_became_active":"2023-08-11T15:44:19.151140+0000","last_became_peered":"2023-08-11T15:44:19.151140+0000","last_unstale":"2023-08-11T15:44:54.196145+0000","last_undegraded":"2023-08-11T15:44:54.196145+0000","last_fullsized":"2023-08-11T15:44:54.196145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:08:09.404210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398895+0000","last_change":"2023-08-11T15:44:19.862255+0000","last_active":"2023-08-11T15:44:25.398895+0000","last_peered":"2023-08-11T15:44:25.398895+0000","last_clean":"2023-08-11T15:44:25.398895+0000","last_became_active":"2023-08-11T15:44:19.861907+0000","last_became_peered":"2023-08-11T15:44:19.861907+0000","last_unstale":"2023-08-11T15:44:25.398895+0000","last_undegraded":"2023-08-11T15:44:25.398895+0000","last_fullsized":"2023-08-11T15:44:25.398895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:03.436493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178469+0000","last_change":"2023-08-11T15:44:24.300513+0000","last_active":"2023-08-11T15:44:25.178469+0000","last_peered":"2023-08-11T15:44:25.178469+0000","last_clean":"2023-08-11T15:44:25.178469+0000","last_became_active":"2023-08-11T15:44:24.300074+0000","last_became_peered":"2023-08-11T15:44:24.300074+0000","last_unstale":"2023-08-11T15:44:25.178469+0000","last_undegraded":"2023-08-11T15:44:25.178469+0000","last_fullsized":"2023-08-11T15:44:25.178469+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:16:21.212112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181835+0000","last_change":"2023-08-11T15:44:19.153794+0000","last_active":"2023-08-11T15:44:25.181835+0000","last_peered":"2023-08-11T15:44:25.181835+0000","last_clean":"2023-08-11T15:44:25.181835+0000","last_became_active":"2023-08-11T15:44:19.153392+0000","last_became_peered":"2023-08-11T15:44:19.153392+0000","last_unstale":"2023-08-11T15:44:25.181835+0000","last_undegraded":"2023-08-11T15:44:25.181835+0000","last_fullsized":"2023-08-11T15:44:25.181835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:47.991902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179190+0000","last_change":"2023-08-11T15:44:19.859799+0000","last_active":"2023-08-11T15:44:25.179190+0000","last_peered":"2023-08-11T15:44:25.179190+0000","last_clean":"2023-08-11T15:44:25.179190+0000","last_became_active":"2023-08-11T15:44:19.859411+0000","last_became_peered":"2023-08-11T15:44:19.859411+0000","last_unstale":"2023-08-11T15:44:25.179190+0000","last_undegraded":"2023-08-11T15:44:25.179190+0000","last_fullsized":"2023-08-11T15:44:25.179190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:19.443644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130327+0000","last_change":"2023-08-11T15:44:24.190272+0000","last_active":"2023-08-11T15:44:27.130327+0000","last_peered":"2023-08-11T15:44:27.130327+0000","last_clean":"2023-08-11T15:44:27.130327+0000","last_became_active":"2023-08-11T15:44:24.189849+0000","last_became_peered":"2023-08-11T15:44:24.189849+0000","last_unstale":"2023-08-11T15:44:27.130327+0000","last_undegraded":"2023-08-11T15:44:27.130327+0000","last_fullsized":"2023-08-11T15:44:27.130327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:26:26.603082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196757+0000","last_change":"2023-08-11T15:44:19.154985+0000","last_active":"2023-08-11T15:44:25.196757+0000","last_peered":"2023-08-11T15:44:25.196757+0000","last_clean":"2023-08-11T15:44:25.196757+0000","last_became_active":"2023-08-11T15:44:19.154694+0000","last_became_peered":"2023-08-11T15:44:19.154694+0000","last_unstale":"2023-08-11T15:44:25.196757+0000","last_undegraded":"2023-08-11T15:44:25.196757+0000","last_fullsized":"2023-08-11T15:44:25.196757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:15.737671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241127+0000","last_change":"2023-08-11T15:44:19.852247+0000","last_active":"2023-08-11T15:44:26.241127+0000","last_peered":"2023-08-11T15:44:26.241127+0000","last_clean":"2023-08-11T15:44:26.241127+0000","last_became_active":"2023-08-11T15:44:19.851456+0000","last_became_peered":"2023-08-11T15:44:19.851456+0000","last_unstale":"2023-08-11T15:44:26.241127+0000","last_undegraded":"2023-08-11T15:44:26.241127+0000","last_fullsized":"2023-08-11T15:44:26.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:57:45.705759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196509+0000","last_change":"2023-08-11T15:44:24.300440+0000","last_active":"2023-08-11T15:44:25.196509+0000","last_peered":"2023-08-11T15:44:25.196509+0000","last_clean":"2023-08-11T15:44:25.196509+0000","last_became_active":"2023-08-11T15:44:24.299981+0000","last_became_peered":"2023-08-11T15:44:24.299981+0000","last_unstale":"2023-08-11T15:44:25.196509+0000","last_undegraded":"2023-08-11T15:44:25.196509+0000","last_fullsized":"2023-08-11T15:44:25.196509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:24.190521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496438+0000","last_change":"2023-08-11T15:44:19.151015+0000","last_active":"2023-08-11T15:44:27.496438+0000","last_peered":"2023-08-11T15:44:27.496438+0000","last_clean":"2023-08-11T15:44:27.496438+0000","last_became_active":"2023-08-11T15:44:19.150677+0000","last_became_peered":"2023-08-11T15:44:19.150677+0000","last_unstale":"2023-08-11T15:44:27.496438+0000","last_undegraded":"2023-08-11T15:44:27.496438+0000","last_fullsized":"2023-08-11T15:44:27.496438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:38:34.368253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.195188+0000","last_change":"2023-08-11T15:44:19.861033+0000","last_active":"2023-08-11T15:44:54.195188+0000","last_peered":"2023-08-11T15:44:54.195188+0000","last_clean":"2023-08-11T15:44:54.195188+0000","last_became_active":"2023-08-11T15:44:19.860643+0000","last_became_peered":"2023-08-11T15:44:19.860643+0000","last_unstale":"2023-08-11T15:44:54.195188+0000","last_undegraded":"2023-08-11T15:44:54.195188+0000","last_fullsized":"2023-08-11T15:44:54.195188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:56:51.466526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181930+0000","last_change":"2023-08-11T15:44:24.199389+0000","last_active":"2023-08-11T15:44:25.181930+0000","last_peered":"2023-08-11T15:44:25.181930+0000","last_clean":"2023-08-11T15:44:25.181930+0000","last_became_active":"2023-08-11T15:44:24.196226+0000","last_became_peered":"2023-08-11T15:44:24.196226+0000","last_unstale":"2023-08-11T15:44:25.181930+0000","last_undegraded":"2023-08-11T15:44:25.181930+0000","last_fullsized":"2023-08-11T15:44:25.181930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:03.253768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178850+0000","last_change":"2023-08-11T15:44:19.152895+0000","last_active":"2023-08-11T15:44:25.178850+0000","last_peered":"2023-08-11T15:44:25.178850+0000","last_clean":"2023-08-11T15:44:25.178850+0000","last_became_active":"2023-08-11T15:44:19.152577+0000","last_became_peered":"2023-08-11T15:44:19.152577+0000","last_unstale":"2023-08-11T15:44:25.178850+0000","last_undegraded":"2023-08-11T15:44:25.178850+0000","last_fullsized":"2023-08-11T15:44:25.178850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:41:12.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183940+0000","last_change":"2023-08-11T15:44:19.861101+0000","last_active":"2023-08-11T15:44:25.183940+0000","last_peered":"2023-08-11T15:44:25.183940+0000","last_clean":"2023-08-11T15:44:25.183940+0000","last_became_active":"2023-08-11T15:44:19.860529+0000","last_became_peered":"2023-08-11T15:44:19.860529+0000","last_unstale":"2023-08-11T15:44:25.183940+0000","last_undegraded":"2023-08-11T15:44:25.183940+0000","last_fullsized":"2023-08-11T15:44:25.183940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:25.463560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196480+0000","last_change":"2023-08-11T15:44:24.202856+0000","last_active":"2023-08-11T15:44:25.196480+0000","last_peered":"2023-08-11T15:44:25.196480+0000","last_clean":"2023-08-11T15:44:25.196480+0000","last_became_active":"2023-08-11T15:44:24.202317+0000","last_became_peered":"2023-08-11T15:44:24.202317+0000","last_unstale":"2023-08-11T15:44:25.196480+0000","last_undegraded":"2023-08-11T15:44:25.196480+0000","last_fullsized":"2023-08-11T15:44:25.196480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:12:44.969687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130438+0000","last_change":"2023-08-11T15:44:19.148711+0000","last_active":"2023-08-11T15:44:27.130438+0000","last_peered":"2023-08-11T15:44:27.130438+0000","last_clean":"2023-08-11T15:44:27.130438+0000","last_became_active":"2023-08-11T15:44:19.148361+0000","last_became_peered":"2023-08-11T15:44:19.148361+0000","last_unstale":"2023-08-11T15:44:27.130438+0000","last_undegraded":"2023-08-11T15:44:27.130438+0000","last_fullsized":"2023-08-11T15:44:27.130438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:45.703207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178814+0000","last_change":"2023-08-11T15:44:19.859562+0000","last_active":"2023-08-11T15:44:25.178814+0000","last_peered":"2023-08-11T15:44:25.178814+0000","last_clean":"2023-08-11T15:44:25.178814+0000","last_became_active":"2023-08-11T15:44:19.859243+0000","last_became_peered":"2023-08-11T15:44:19.859243+0000","last_unstale":"2023-08-11T15:44:25.178814+0000","last_undegraded":"2023-08-11T15:44:25.178814+0000","last_fullsized":"2023-08-11T15:44:25.178814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:50.690599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196555+0000","last_change":"2023-08-11T15:44:24.300317+0000","last_active":"2023-08-11T15:44:25.196555+0000","last_peered":"2023-08-11T15:44:25.196555+0000","last_clean":"2023-08-11T15:44:25.196555+0000","last_became_active":"2023-08-11T15:44:24.299841+0000","last_became_peered":"2023-08-11T15:44:24.299841+0000","last_unstale":"2023-08-11T15:44:25.196555+0000","last_undegraded":"2023-08-11T15:44:25.196555+0000","last_fullsized":"2023-08-11T15:44:25.196555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:16.800305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194494+0000","last_change":"2023-08-11T15:44:19.149111+0000","last_active":"2023-08-11T15:44:54.194494+0000","last_peered":"2023-08-11T15:44:54.194494+0000","last_clean":"2023-08-11T15:44:54.194494+0000","last_became_active":"2023-08-11T15:44:19.148173+0000","last_became_peered":"2023-08-11T15:44:19.148173+0000","last_unstale":"2023-08-11T15:44:54.194494+0000","last_undegraded":"2023-08-11T15:44:54.194494+0000","last_fullsized":"2023-08-11T15:44:54.194494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:51.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189482+0000","last_change":"2023-08-11T15:44:19.863724+0000","last_active":"2023-08-11T15:44:54.189482+0000","last_peered":"2023-08-11T15:44:54.189482+0000","last_clean":"2023-08-11T15:44:54.189482+0000","last_became_active":"2023-08-11T15:44:19.863268+0000","last_became_peered":"2023-08-11T15:44:19.863268+0000","last_unstale":"2023-08-11T15:44:54.189482+0000","last_undegraded":"2023-08-11T15:44:54.189482+0000","last_fullsized":"2023-08-11T15:44:54.189482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:27:34.481889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497528+0000","last_change":"2023-08-11T15:44:24.191301+0000","last_active":"2023-08-11T15:44:27.497528+0000","last_peered":"2023-08-11T15:44:27.497528+0000","last_clean":"2023-08-11T15:44:27.497528+0000","last_became_active":"2023-08-11T15:44:24.190686+0000","last_became_peered":"2023-08-11T15:44:24.190686+0000","last_unstale":"2023-08-11T15:44:27.497528+0000","last_undegraded":"2023-08-11T15:44:27.497528+0000","last_fullsized":"2023-08-11T15:44:27.497528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:29:52.896841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130772+0000","last_change":"2023-08-11T15:44:19.147756+0000","last_active":"2023-08-11T15:44:27.130772+0000","last_peered":"2023-08-11T15:44:27.130772+0000","last_clean":"2023-08-11T15:44:27.130772+0000","last_became_active":"2023-08-11T15:44:19.147426+0000","last_became_peered":"2023-08-11T15:44:19.147426+0000","last_unstale":"2023-08-11T15:44:27.130772+0000","last_undegraded":"2023-08-11T15:44:27.130772+0000","last_fullsized":"2023-08-11T15:44:27.130772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:45.863313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190289+0000","last_change":"2023-08-11T15:44:19.863354+0000","last_active":"2023-08-11T15:44:54.190289+0000","last_peered":"2023-08-11T15:44:54.190289+0000","last_clean":"2023-08-11T15:44:54.190289+0000","last_became_active":"2023-08-11T15:44:19.862700+0000","last_became_peered":"2023-08-11T15:44:19.862700+0000","last_unstale":"2023-08-11T15:44:54.190289+0000","last_undegraded":"2023-08-11T15:44:54.190289+0000","last_fullsized":"2023-08-11T15:44:54.190289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:14:44.778310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240828+0000","last_change":"2023-08-11T15:44:24.204657+0000","last_active":"2023-08-11T15:44:26.240828+0000","last_peered":"2023-08-11T15:44:26.240828+0000","last_clean":"2023-08-11T15:44:26.240828+0000","last_became_active":"2023-08-11T15:44:24.204108+0000","last_became_peered":"2023-08-11T15:44:24.204108+0000","last_unstale":"2023-08-11T15:44:26.240828+0000","last_undegraded":"2023-08-11T15:44:26.240828+0000","last_fullsized":"2023-08-11T15:44:26.240828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:17.431253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044145+0000","last_change":"2023-08-11T15:44:19.147014+0000","last_active":"2023-08-11T15:44:27.044145+0000","last_peered":"2023-08-11T15:44:27.044145+0000","last_clean":"2023-08-11T15:44:27.044145+0000","last_became_active":"2023-08-11T15:44:19.146720+0000","last_became_peered":"2023-08-11T15:44:19.146720+0000","last_unstale":"2023-08-11T15:44:27.044145+0000","last_undegraded":"2023-08-11T15:44:27.044145+0000","last_fullsized":"2023-08-11T15:44:27.044145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:42:56.620766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.498380+0000","last_change":"2023-08-11T15:44:54.190492+0000","last_active":"2023-08-11T15:44:54.190492+0000","last_peered":"2023-08-11T15:44:27.498380+0000","last_clean":"2023-08-11T15:44:27.498380+0000","last_became_active":"2023-08-11T15:44:19.853078+0000","last_became_peered":"2023-08-11T15:44:19.853078+0000","last_unstale":"2023-08-11T15:44:27.498380+0000","last_undegraded":"2023-08-11T15:44:27.498380+0000","last_fullsized":"2023-08-11T15:44:27.498380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043195+0000","last_change":"2023-08-11T15:44:24.192709+0000","last_active":"2023-08-11T15:44:27.043195+0000","last_peered":"2023-08-11T15:44:27.043195+0000","last_clean":"2023-08-11T15:44:27.043195+0000","last_became_active":"2023-08-11T15:44:24.192297+0000","last_became_peered":"2023-08-11T15:44:24.192297+0000","last_unstale":"2023-08-11T15:44:27.043195+0000","last_undegraded":"2023-08-11T15:44:27.043195+0000","last_fullsized":"2023-08-11T15:44:27.043195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:39.723615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399018+0000","last_change":"2023-08-11T15:44:19.158301+0000","last_active":"2023-08-11T15:44:25.399018+0000","last_peered":"2023-08-11T15:44:25.399018+0000","last_clean":"2023-08-11T15:44:25.399018+0000","last_became_active":"2023-08-11T15:44:19.157990+0000","last_became_peered":"2023-08-11T15:44:19.157990+0000","last_unstale":"2023-08-11T15:44:25.399018+0000","last_undegraded":"2023-08-11T15:44:25.399018+0000","last_fullsized":"2023-08-11T15:44:25.399018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:02.696501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189944+0000","last_change":"2023-08-11T15:44:19.857357+0000","last_active":"2023-08-11T15:44:54.189944+0000","last_peered":"2023-08-11T15:44:54.189944+0000","last_clean":"2023-08-11T15:44:54.189944+0000","last_became_active":"2023-08-11T15:44:19.857020+0000","last_became_peered":"2023-08-11T15:44:19.857020+0000","last_unstale":"2023-08-11T15:44:54.189944+0000","last_undegraded":"2023-08-11T15:44:54.189944+0000","last_fullsized":"2023-08-11T15:44:54.189944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:37.243586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183978+0000","last_change":"2023-08-11T15:44:19.863314+0000","last_active":"2023-08-11T15:44:25.183978+0000","last_peered":"2023-08-11T15:44:25.183978+0000","last_clean":"2023-08-11T15:44:25.183978+0000","last_became_active":"2023-08-11T15:44:19.862964+0000","last_became_peered":"2023-08-11T15:44:19.862964+0000","last_unstale":"2023-08-11T15:44:25.183978+0000","last_undegraded":"2023-08-11T15:44:25.183978+0000","last_fullsized":"2023-08-11T15:44:25.183978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:16.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130670+0000","last_change":"2023-08-11T15:44:19.850575+0000","last_active":"2023-08-11T15:44:27.130670+0000","last_peered":"2023-08-11T15:44:27.130670+0000","last_clean":"2023-08-11T15:44:27.130670+0000","last_became_active":"2023-08-11T15:44:19.850231+0000","last_became_peered":"2023-08-11T15:44:19.850231+0000","last_unstale":"2023-08-11T15:44:27.130670+0000","last_undegraded":"2023-08-11T15:44:27.130670+0000","last_fullsized":"2023-08-11T15:44:27.130670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:21.005350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130303+0000","last_change":"2023-08-11T15:44:19.857387+0000","last_active":"2023-08-11T15:44:27.130303+0000","last_peered":"2023-08-11T15:44:27.130303+0000","last_clean":"2023-08-11T15:44:27.130303+0000","last_became_active":"2023-08-11T15:44:19.856830+0000","last_became_peered":"2023-08-11T15:44:19.856830+0000","last_unstale":"2023-08-11T15:44:27.130303+0000","last_undegraded":"2023-08-11T15:44:27.130303+0000","last_fullsized":"2023-08-11T15:44:27.130303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:42.512448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241040+0000","last_change":"2023-08-11T15:44:19.868112+0000","last_active":"2023-08-11T15:44:26.241040+0000","last_peered":"2023-08-11T15:44:26.241040+0000","last_clean":"2023-08-11T15:44:26.241040+0000","last_became_active":"2023-08-11T15:44:19.867610+0000","last_became_peered":"2023-08-11T15:44:19.867610+0000","last_unstale":"2023-08-11T15:44:26.241040+0000","last_undegraded":"2023-08-11T15:44:26.241040+0000","last_fullsized":"2023-08-11T15:44:26.241040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:18.810119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398997+0000","last_change":"2023-08-11T15:44:19.858349+0000","last_active":"2023-08-11T15:44:25.398997+0000","last_peered":"2023-08-11T15:44:25.398997+0000","last_clean":"2023-08-11T15:44:25.398997+0000","last_became_active":"2023-08-11T15:44:19.858041+0000","last_became_peered":"2023-08-11T15:44:19.858041+0000","last_unstale":"2023-08-11T15:44:25.398997+0000","last_undegraded":"2023-08-11T15:44:25.398997+0000","last_fullsized":"2023-08-11T15:44:25.398997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:04.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184464+0000","last_change":"2023-08-11T15:44:19.865383+0000","last_active":"2023-08-11T15:44:25.184464+0000","last_peered":"2023-08-11T15:44:25.184464+0000","last_clean":"2023-08-11T15:44:25.184464+0000","last_became_active":"2023-08-11T15:44:19.865065+0000","last_became_peered":"2023-08-11T15:44:19.865065+0000","last_unstale":"2023-08-11T15:44:25.184464+0000","last_undegraded":"2023-08-11T15:44:25.184464+0000","last_fullsized":"2023-08-11T15:44:25.184464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:39:27.425778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130961+0000","last_change":"2023-08-11T15:44:19.859297+0000","last_active":"2023-08-11T15:44:27.130961+0000","last_peered":"2023-08-11T15:44:27.130961+0000","last_clean":"2023-08-11T15:44:27.130961+0000","last_became_active":"2023-08-11T15:44:19.857517+0000","last_became_peered":"2023-08-11T15:44:19.857517+0000","last_unstale":"2023-08-11T15:44:27.130961+0000","last_undegraded":"2023-08-11T15:44:27.130961+0000","last_fullsized":"2023-08-11T15:44:27.130961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:52:53.104396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179277+0000","last_change":"2023-08-11T15:44:19.850485+0000","last_active":"2023-08-11T15:44:25.179277+0000","last_peered":"2023-08-11T15:44:25.179277+0000","last_clean":"2023-08-11T15:44:25.179277+0000","last_became_active":"2023-08-11T15:44:19.850047+0000","last_became_peered":"2023-08-11T15:44:19.850047+0000","last_unstale":"2023-08-11T15:44:25.179277+0000","last_undegraded":"2023-08-11T15:44:25.179277+0000","last_fullsized":"2023-08-11T15:44:25.179277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:11.380061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043361+0000","last_change":"2023-08-11T15:44:19.861953+0000","last_active":"2023-08-11T15:44:27.043361+0000","last_peered":"2023-08-11T15:44:27.043361+0000","last_clean":"2023-08-11T15:44:27.043361+0000","last_became_active":"2023-08-11T15:44:19.861523+0000","last_became_peered":"2023-08-11T15:44:19.861523+0000","last_unstale":"2023-08-11T15:44:27.043361+0000","last_undegraded":"2023-08-11T15:44:27.043361+0000","last_fullsized":"2023-08-11T15:44:27.043361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:14.229621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178755+0000","last_change":"2023-08-11T15:44:19.849283+0000","last_active":"2023-08-11T15:44:25.178755+0000","last_peered":"2023-08-11T15:44:25.178755+0000","last_clean":"2023-08-11T15:44:25.178755+0000","last_became_active":"2023-08-11T15:44:19.848304+0000","last_became_peered":"2023-08-11T15:44:19.848304+0000","last_unstale":"2023-08-11T15:44:25.178755+0000","last_undegraded":"2023-08-11T15:44:25.178755+0000","last_fullsized":"2023-08-11T15:44:25.178755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:06.449360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497606+0000","last_change":"2023-08-11T15:44:19.853532+0000","last_active":"2023-08-11T15:44:27.497606+0000","last_peered":"2023-08-11T15:44:27.497606+0000","last_clean":"2023-08-11T15:44:27.497606+0000","last_became_active":"2023-08-11T15:44:19.853182+0000","last_became_peered":"2023-08-11T15:44:19.853182+0000","last_unstale":"2023-08-11T15:44:27.497606+0000","last_undegraded":"2023-08-11T15:44:27.497606+0000","last_fullsized":"2023-08-11T15:44:27.497606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:51.245443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398519+0000","last_change":"2023-08-11T15:44:19.860571+0000","last_active":"2023-08-11T15:44:25.398519+0000","last_peered":"2023-08-11T15:44:25.398519+0000","last_clean":"2023-08-11T15:44:25.398519+0000","last_became_active":"2023-08-11T15:44:19.858570+0000","last_became_peered":"2023-08-11T15:44:19.858570+0000","last_unstale":"2023-08-11T15:44:25.398519+0000","last_undegraded":"2023-08-11T15:44:25.398519+0000","last_fullsized":"2023-08-11T15:44:25.398519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:37.343811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184137+0000","last_change":"2023-08-11T15:44:19.865988+0000","last_active":"2023-08-11T15:44:25.184137+0000","last_peered":"2023-08-11T15:44:25.184137+0000","last_clean":"2023-08-11T15:44:25.184137+0000","last_became_active":"2023-08-11T15:44:19.865613+0000","last_became_peered":"2023-08-11T15:44:19.865613+0000","last_unstale":"2023-08-11T15:44:25.184137+0000","last_undegraded":"2023-08-11T15:44:25.184137+0000","last_fullsized":"2023-08-11T15:44:25.184137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:07.472727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":48682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.77400000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1061196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.627}]},{"osd":3,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.503}]},{"osd":4,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38400000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.497}]},{"osd":8,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.61299999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83399999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.627},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.751}]},{"osd":6,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.67100000000000004}]},{"osd":8,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.80300000000000005}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":481002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.75800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72499999999999998,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.85799999999999998,"5min":1.05,"15min":1.05},"last":1.1160000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.435,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99199999999999999,"5min":1.077,"15min":1.077},"last":1.145},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0980000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.96899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95399999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":1.1859999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.5819999999999999,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":1.044}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.1299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1075775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.45500000000000002,"15min":0.44800000000000001},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.628},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Aug 11 15:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.79300000000000004,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.498},"max":{"1min":4.8890000000000002,"5min":4.8890000000000002,"15min":4.8890000000000002},"last":0.88}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":2.077,"15min":2.077},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.008,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.581,"15min":1.581},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92500000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.7,"15min":1.7},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0269999999999999,"5min":1.22,"15min":1.22},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1319999999999999,"5min":1.623,"15min":1.623},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.61799999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.63,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.90000000000000002,"15min":0.89200000000000002},"min":{"1min":0.64500000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.85499999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.98399999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.95199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":470965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.76600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88600000000000001,"5min":4.9020000000000001,"15min":4.9020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.45900000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.62}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80800000000000005,"5min":1.593,"15min":1.593},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 15:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.024,"15min":1.024},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74099999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":1.05,"15min":1.05},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.90100000000000002}]},{"osd":8,"last update":"Fri Aug 11 15:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67700000000000005,"15min":0.66600000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1180000000000001,"5min":1.298,"15min":1.298},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.5409999999999999,"5min":6.0609999999999999,"15min":6.0609999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.087,"5min":6.101,"15min":6.101},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.621,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.441,"15min":0.433},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46100000000000002,"15min":0.44700000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.65300000000000002,"15min":0.61299999999999999},"min":{"1min":0.56000000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.54300000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.056,"5min":1.175,"15min":1.175},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.032,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.74399999999999999,"15min":0.72799999999999998},"min":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.78900000000000003,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.85799999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1779999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.149,"5min":1.276,"15min":1.276},"last":1.0489999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.51400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.81999999999999995}]},{"osd":11,"last update":"Fri Aug 11 15:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.016}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":470999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54600000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80300000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.44},"min":{"1min":0.29699999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69499999999999995}]},{"osd":8,"last update":"Fri Aug 11 15:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.879},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.97499999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.95999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":467227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.47699999999999998,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68100000000000005,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.746}]},{"osd":2,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55600000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94399999999999995,"5min":1.105,"15min":1.105},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.53800000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.72399999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.58799999999999997,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Aug 11 15:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.629},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.73599999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95099999999999996,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.266,"5min":5.266,"15min":5.266},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.51000000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.84799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.317},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.35999999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83599999999999997,"5min":1.552,"15min":1.552},"last":0.626},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":1.502,"15min":1.502},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47499999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.016,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93899999999999995,"5min":1.526,"15min":1.526},"last":0.68999999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1057342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79500000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60599999999999998,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83299999999999996,"5min":5.359,"15min":5.359},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.434,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.872,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.443},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77000000000000002,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.36899999999999999},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41999999999999998,"15min":0.40400000000000003},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.73099999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.032,"15min":1.032},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82599999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]},{"osd":8,"last update":"Fri Aug 11 15:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.71599999999999997}]},{"osd":10,"last update":"Fri Aug 11 15:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":21257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:44:56.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2023-08-11T15:44:56.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 21, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-08-11T15:44:27.496972+0000', 'last_change': '2023-08-11T15:44:54.190469+0000', 'last_active': '2023-08-11T15:44:54.190469+0000', 'last_peered': '2023-08-11T15:44:27.496972+0000', 'last_clean': '2023-08-11T15:44:27.496972+0000', 'last_became_active': '2023-08-11T15:44:19.144155+0000', 'last_became_peered': '2023-08-11T15:44:19.144155+0000', 'last_unstale': '2023-08-11T15:44:27.496972+0000', 'last_undegraded': '2023-08-11T15:44:27.496972+0000', 'last_fullsized': '2023-08-11T15:44:27.496972+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-11T15:44:18.081545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-11T15:44:18.081545+0000', 'last_clean_scrub_stamp': '2023-08-11T15:44:18.081545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1, 4], 'acting': [7, 1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2, 4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-11T15:44:56.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-08-11T15:44:56.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-08-11T15:44:27.497633+0000', 'last_change': '2023-08-11T15:44:54.195412+0000', 'last_active': '2023-08-11T15:44:54.195412+0000', 'last_peered': '2023-08-11T15:44:27.497633+0000', 'last_clean': '2023-08-11T15:44:27.497633+0000', 'last_became_active': '2023-08-11T15:44:24.300268+0000', 'last_became_peered': '2023-08-11T15:44:24.300268+0000', 'last_unstale': '2023-08-11T15:44:27.497633+0000', 'last_undegraded': '2023-08-11T15:44:27.497633+0000', 'last_fullsized': '2023-08-11T15:44:27.497633+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 86, 'last_epoch_clean': 87, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-11T15:44:23.170471+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-11T15:44:23.170471+0000', 'last_clean_scrub_stamp': '2023-08-11T15:44:23.170471+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 10, 11, 1], 'acting': [8, 10, 11, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 9, 10, 11], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-08-11T15:44:56.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-08-11T15:44:56.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 19, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-08-11T15:44:27.498380+0000', 'last_change': '2023-08-11T15:44:54.190492+0000', 'last_active': '2023-08-11T15:44:54.190492+0000', 'last_peered': '2023-08-11T15:44:27.498380+0000', 'last_clean': '2023-08-11T15:44:27.498380+0000', 'last_became_active': '2023-08-11T15:44:19.853078+0000', 'last_became_peered': '2023-08-11T15:44:19.853078+0000', 'last_unstale': '2023-08-11T15:44:27.498380+0000', 'last_undegraded': '2023-08-11T15:44:27.498380+0000', 'last_fullsized': '2023-08-11T15:44:27.498380+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 83, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-11T15:44:19.122192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-11T15:44:19.122192+0000', 'last_clean_scrub_stamp': '2023-08-11T15:44:19.122192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 9, 5], 'acting': [7, 9, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 5, 9], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-11T15:44:56.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-11T15:44:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 -- 172.21.15.32:0/1392153314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31641087e0 msgr2=0x7f316410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 --2- 172.21.15.32:0/1392153314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31641087e0 0x7f316410ac70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f315c00b0a0 tx=0x7f315c00b3b0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 -- 172.21.15.32:0/1392153314 shutdown_connections 2023-08-11T15:44:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 --2- 172.21.15.32:0/1392153314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31641087e0 0x7f316410ac70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 --2- 172.21.15.32:0/1392153314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164074960 0x7f3164072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 --2- 172.21.15.32:0/1392153314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3164073380 0x7f3164073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 -- 172.21.15.32:0/1392153314 >> 172.21.15.32:0/1392153314 conn(0x7f31640fbe50 msgr2=0x7f31640fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 -- 172.21.15.32:0/1392153314 shutdown_connections 2023-08-11T15:44:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.953+0000 7f316adff700 1 -- 172.21.15.32:0/1392153314 wait complete. 2023-08-11T15:44:56.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 Processor -- start 2023-08-11T15:44:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 -- start start 2023-08-11T15:44:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3164073380 0x7f31641988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3164074960 0x7f3164198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31641087e0 0x7f316419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31641a0460 con 0x7f3164073380 2023-08-11T15:44:56.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31641a05a0 con 0x7f3164074960 2023-08-11T15:44:56.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.954+0000 7f316adff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31641a0730 con 0x7f31641087e0 2023-08-11T15:44:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3164073380 0x7f31641988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3164073380 0x7f31641988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47114/0 (socket says 172.21.15.32:47114) 2023-08-11T15:44:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 -- 172.21.15.32:0/3895105932 learned_addr learned my addr 172.21.15.32:0/3895105932 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3163fff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3164074960 0x7f3164198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f316939c700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31641087e0 0x7f316419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 -- 172.21.15.32:0/3895105932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31641087e0 msgr2=0x7f316419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31641087e0 0x7f316419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 -- 172.21.15.32:0/3895105932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3164074960 msgr2=0x7f3164198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3164074960 0x7f3164198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 -- 172.21.15.32:0/3895105932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c009d00 con 0x7f3164073380 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.955+0000 7f3168b9b700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3164073380 0x7f31641988e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f315800e9a0 tx=0x7f315800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.956+0000 7f3161ffb700 1 -- 172.21.15.32:0/3895105932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158009790 con 0x7f3164073380 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.956+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31641a0a10 con 0x7f3164073380 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.956+0000 7f3161ffb700 1 -- 172.21.15.32:0/3895105932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3158010430 con 0x7f3164073380 2023-08-11T15:44:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.956+0000 7f3161ffb700 1 -- 172.21.15.32:0/3895105932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158018680 con 0x7f3164073380 2023-08-11T15:44:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.956+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31641a0f70 con 0x7f3164073380 2023-08-11T15:44:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.957+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164106e90 con 0x7f3164073380 2023-08-11T15:44:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.959+0000 7f3161ffb700 1 -- 172.21.15.32:0/3895105932 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f31580105a0 con 0x7f3164073380 2023-08-11T15:44:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.960+0000 7f3161ffb700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31540a00a0 0x7f31540a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.960+0000 7f3161ffb700 1 -- 172.21.15.32:0/3895105932 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3158014070 con 0x7f3164073380 2023-08-11T15:44:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.961+0000 7f3163fff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31540a00a0 0x7f31540a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.961+0000 7f3163fff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31540a00a0 0x7f31540a2560 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3164074680 tx=0x7f3150009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:56.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:56.964+0000 7f3161ffb700 1 -- 172.21.15.32:0/3895105932 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3158051c30 con 0x7f3164073380 2023-08-11T15:44:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.262+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f316410e5f0 con 0x7f3164073380 2023-08-11T15:44:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.263+0000 7f3161ffb700 1 -- 172.21.15.32:0/3895105932 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2199 (secure 0 0 0) 0x7f3158095140 con 0x7f3164073380 2023-08-11T15:44:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:57.266 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":367,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":90,"num_osds":12,"num_up_osds":12,"osd_up_since":1691768611,"num_in_osds":12,"osd_in_since":1691768594,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":126},{"state_name":"peering","count":3}],"num_pgs":129,"num_pools":4,"num_objects":25,"data_bytes":595131,"bytes_used":345497600,"bytes_avail":1151594930176,"bytes_total":1151940427776,"inactive_pgs_ratio":0.023255813866853714,"read_bytes_sec":1535,"write_bytes_sec":127,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":21,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:standby-replay","gid":24566},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24578}],"up:standby":10},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.32:8443/"}},"servicemap":{"epoch":23,"modified":"2023-08-11T15:44:56.075843+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-11T15:44:57.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31540a00a0 msgr2=0x7f31540a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31540a00a0 0x7f31540a2560 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3164074680 tx=0x7f3150009260 comp rx=0 tx=0).stop 2023-08-11T15:44:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3164073380 msgr2=0x7f31641988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3164073380 0x7f31641988e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f315800e9a0 tx=0x7f315800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 shutdown_connections 2023-08-11T15:44:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31540a00a0 0x7f31540a2560 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3164073380 0x7f31641988e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31641087e0 0x7f316419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 --2- 172.21.15.32:0/3895105932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3164074960 0x7f3164198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 >> 172.21.15.32:0/3895105932 conn(0x7f31640fbe50 msgr2=0x7f316410aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 shutdown_connections 2023-08-11T15:44:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.265+0000 7f316adff700 1 -- 172.21.15.32:0/3895105932 wait complete. 2023-08-11T15:44:57.289 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-11T15:44:57.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:44:57.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:57 smithi134 ceph-mon[130447]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:57 smithi134 ceph-mon[130447]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 127 B/s wr, 1 op/s 2023-08-11T15:44:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:57 smithi134 ceph-mon[130447]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:57.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 -- 172.21.15.32:0/3560465256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62481024b0 msgr2=0x7f6248108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:57.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 --2- 172.21.15.32:0/3560465256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62481024b0 0x7f6248108d60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f624000b0d0 tx=0x7f624000b3e0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 -- 172.21.15.32:0/3560465256 shutdown_connections 2023-08-11T15:44:57.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 --2- 172.21.15.32:0/3560465256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62481024b0 0x7f6248108d60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 --2- 172.21.15.32:0/3560465256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6248100920 0x7f6248100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 --2- 172.21.15.32:0/3560465256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6248101b10 0x7f6248101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 -- 172.21.15.32:0/3560465256 >> 172.21.15.32:0/3560465256 conn(0x7f62480fbe50 msgr2=0x7f62480fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 -- 172.21.15.32:0/3560465256 shutdown_connections 2023-08-11T15:44:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.491+0000 7f624ddff700 1 -- 172.21.15.32:0/3560465256 wait complete. 2023-08-11T15:44:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.493+0000 7f624ddff700 1 Processor -- start 2023-08-11T15:44:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.493+0000 7f624ddff700 1 -- start start 2023-08-11T15:44:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.493+0000 7f624ddff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6248100920 0x7f62480746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:57.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.493+0000 7f624ddff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6248101b10 0x7f6248072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:57.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.493+0000 7f624ddff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62481024b0 0x7f6248073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:57.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f624ddff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480739f0 con 0x7f6248101b10 2023-08-11T15:44:57.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f624ddff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6248073b60 con 0x7f6248100920 2023-08-11T15:44:57.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f624ddff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62481a0740 con 0x7f62481024b0 2023-08-11T15:44:57.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6248101b10 0x7f6248072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:57.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6248101b10 0x7f6248072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47126/0 (socket says 172.21.15.32:47126) 2023-08-11T15:44:57.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 -- 172.21.15.32:0/2176387687 learned_addr learned my addr 172.21.15.32:0/2176387687 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:44:57.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f62477fe700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6248100920 0x7f62480746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:57.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6247fff700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62481024b0 0x7f6248073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:57.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 -- 172.21.15.32:0/2176387687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62481024b0 msgr2=0x7f6248073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:57.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62481024b0 0x7f6248073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 -- 172.21.15.32:0/2176387687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6248100920 msgr2=0x7f62480746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:57.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6248100920 0x7f62480746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.494+0000 7f6246ffd700 1 -- 172.21.15.32:0/2176387687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240009d00 con 0x7f6248101b10 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.495+0000 7f6246ffd700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6248101b10 0x7f6248072d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e9a0 tx=0x7f623c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.495+0000 7f6244ff9700 1 -- 172.21.15.32:0/2176387687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c009790 con 0x7f6248101b10 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.495+0000 7f6244ff9700 1 -- 172.21.15.32:0/2176387687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f623c00cbd0 con 0x7f6248101b10 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.495+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62481a0a20 con 0x7f6248101b10 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.495+0000 7f6244ff9700 1 -- 172.21.15.32:0/2176387687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c018680 con 0x7f6248101b10 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.495+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62481a0f30 con 0x7f6248101b10 2023-08-11T15:44:57.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.496+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624810c190 con 0x7f6248101b10 2023-08-11T15:44:57.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.498+0000 7f6244ff9700 1 -- 172.21.15.32:0/2176387687 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f623c00cd40 con 0x7f6248101b10 2023-08-11T15:44:57.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.499+0000 7f6244ff9700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f62300a0020 0x7f62300a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:44:57.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.500+0000 7f6244ff9700 1 -- 172.21.15.32:0/2176387687 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f623c014070 con 0x7f6248101b10 2023-08-11T15:44:57.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.500+0000 7f62477fe700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f62300a0020 0x7f62300a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:44:57.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.500+0000 7f62477fe700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f62300a0020 0x7f62300a24e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6248101970 tx=0x7f6238009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:44:57.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.503+0000 7f6244ff9700 1 -- 172.21.15.32:0/2176387687 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f623c051c30 con 0x7f6248101b10 2023-08-11T15:44:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:57 smithi032 ceph-mon[97103]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:57.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:57 smithi032 ceph-mon[97103]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 127 B/s wr, 1 op/s 2023-08-11T15:44:57.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:57 smithi032 ceph-mon[97103]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:57.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:57 smithi192 ceph-mon[130806]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:57.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:57 smithi192 ceph-mon[130806]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 127 B/s wr, 1 op/s 2023-08-11T15:44:57.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:57 smithi192 ceph-mon[130806]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:57.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.717+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6248060d60 con 0x7f62300a0020 2023-08-11T15:44:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.730+0000 7f6244ff9700 1 -- 172.21.15.32:0/2176387687 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422261 (secure 0 0 0) 0x7f6248060d60 con 0x7f62300a0020 2023-08-11T15:44:57.740 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:44:57.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f62300a0020 msgr2=0x7f62300a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:57.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f62300a0020 0x7f62300a24e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6248101970 tx=0x7f6238009260 comp rx=0 tx=0).stop 2023-08-11T15:44:57.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6248101b10 msgr2=0x7f6248072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:44:57.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6248101b10 0x7f6248072d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e9a0 tx=0x7f623c00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 shutdown_connections 2023-08-11T15:44:57.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f62300a0020 0x7f62300a24e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62481024b0 0x7f6248073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6248100920 0x7f62480746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 --2- 172.21.15.32:0/2176387687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6248101b10 0x7f6248072d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:44:57.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 >> 172.21.15.32:0/2176387687 conn(0x7f62480fbe50 msgr2=0x7f62480fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:44:57.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 shutdown_connections 2023-08-11T15:44:57.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:44:57.735+0000 7f624ddff700 1 -- 172.21.15.32:0/2176387687 wait complete. 2023-08-11T15:44:57.748 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-08-11T15:44:57.756 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-08-11T15:44:56.075793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595131,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":147,"num_read_kb":131,"num_write":231,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337400,"kb_used_data":13148,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604424,"statfs":{"total":1151940427776,"available":1151594930176,"internally_reserved":0,"allocated":13463552,"data_stored":7420815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002474"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184488+0000","last_change":"2023-08-11T15:44:19.862927+0000","last_active":"2023-08-11T15:44:25.184488+0000","last_peered":"2023-08-11T15:44:25.184488+0000","last_clean":"2023-08-11T15:44:25.184488+0000","last_became_active":"2023-08-11T15:44:19.862626+0000","last_became_peered":"2023-08-11T15:44:19.862626+0000","last_unstale":"2023-08-11T15:44:25.184488+0000","last_undegraded":"2023-08-11T15:44:25.184488+0000","last_fullsized":"2023-08-11T15:44:25.184488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:58:44.417078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241036+0000","last_change":"2023-08-11T15:44:19.852455+0000","last_active":"2023-08-11T15:44:26.241036+0000","last_peered":"2023-08-11T15:44:26.241036+0000","last_clean":"2023-08-11T15:44:26.241036+0000","last_became_active":"2023-08-11T15:44:19.851897+0000","last_became_peered":"2023-08-11T15:44:19.851897+0000","last_unstale":"2023-08-11T15:44:26.241036+0000","last_undegraded":"2023-08-11T15:44:26.241036+0000","last_fullsized":"2023-08-11T15:44:26.241036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:20.816842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241013+0000","last_change":"2023-08-11T15:44:19.855232+0000","last_active":"2023-08-11T15:44:26.241013+0000","last_peered":"2023-08-11T15:44:26.241013+0000","last_clean":"2023-08-11T15:44:26.241013+0000","last_became_active":"2023-08-11T15:44:19.854940+0000","last_became_peered":"2023-08-11T15:44:19.854940+0000","last_unstale":"2023-08-11T15:44:26.241013+0000","last_undegraded":"2023-08-11T15:44:26.241013+0000","last_fullsized":"2023-08-11T15:44:26.241013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:42:52.881533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196583+0000","last_change":"2023-08-11T15:44:19.856137+0000","last_active":"2023-08-11T15:44:25.196583+0000","last_peered":"2023-08-11T15:44:25.196583+0000","last_clean":"2023-08-11T15:44:25.196583+0000","last_became_active":"2023-08-11T15:44:19.855852+0000","last_became_peered":"2023-08-11T15:44:19.855852+0000","last_unstale":"2023-08-11T15:44:25.196583+0000","last_undegraded":"2023-08-11T15:44:25.196583+0000","last_fullsized":"2023-08-11T15:44:25.196583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:21.362833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194351+0000","last_change":"2023-08-11T15:44:19.862618+0000","last_active":"2023-08-11T15:44:54.194351+0000","last_peered":"2023-08-11T15:44:54.194351+0000","last_clean":"2023-08-11T15:44:54.194351+0000","last_became_active":"2023-08-11T15:44:19.862265+0000","last_became_peered":"2023-08-11T15:44:19.862265+0000","last_unstale":"2023-08-11T15:44:54.194351+0000","last_undegraded":"2023-08-11T15:44:54.194351+0000","last_fullsized":"2023-08-11T15:44:54.194351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:53.783047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909293+0000","last_change":"2023-08-11T15:44:19.857801+0000","last_active":"2023-08-11T15:44:26.909293+0000","last_peered":"2023-08-11T15:44:26.909293+0000","last_clean":"2023-08-11T15:44:26.909293+0000","last_became_active":"2023-08-11T15:44:19.857388+0000","last_became_peered":"2023-08-11T15:44:19.857388+0000","last_unstale":"2023-08-11T15:44:26.909293+0000","last_undegraded":"2023-08-11T15:44:26.909293+0000","last_fullsized":"2023-08-11T15:44:26.909293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:55:45.696116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189336+0000","last_change":"2023-08-11T15:44:19.859334+0000","last_active":"2023-08-11T15:44:54.189336+0000","last_peered":"2023-08-11T15:44:54.189336+0000","last_clean":"2023-08-11T15:44:54.189336+0000","last_became_active":"2023-08-11T15:44:19.859073+0000","last_became_peered":"2023-08-11T15:44:19.859073+0000","last_unstale":"2023-08-11T15:44:54.189336+0000","last_undegraded":"2023-08-11T15:44:54.189336+0000","last_fullsized":"2023-08-11T15:44:54.189336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:37:20.633727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178676+0000","last_change":"2023-08-11T15:44:19.853064+0000","last_active":"2023-08-11T15:44:25.178676+0000","last_peered":"2023-08-11T15:44:25.178676+0000","last_clean":"2023-08-11T15:44:25.178676+0000","last_became_active":"2023-08-11T15:44:19.852736+0000","last_became_peered":"2023-08-11T15:44:19.852736+0000","last_unstale":"2023-08-11T15:44:25.178676+0000","last_undegraded":"2023-08-11T15:44:25.178676+0000","last_fullsized":"2023-08-11T15:44:25.178676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:31.513054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497471+0000","last_change":"2023-08-11T15:44:19.861912+0000","last_active":"2023-08-11T15:44:27.497471+0000","last_peered":"2023-08-11T15:44:27.497471+0000","last_clean":"2023-08-11T15:44:27.497471+0000","last_became_active":"2023-08-11T15:44:19.861374+0000","last_became_peered":"2023-08-11T15:44:19.861374+0000","last_unstale":"2023-08-11T15:44:27.497471+0000","last_undegraded":"2023-08-11T15:44:27.497471+0000","last_fullsized":"2023-08-11T15:44:27.497471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:55.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909786+0000","last_change":"2023-08-11T15:44:19.858862+0000","last_active":"2023-08-11T15:44:26.909786+0000","last_peered":"2023-08-11T15:44:26.909786+0000","last_clean":"2023-08-11T15:44:26.909786+0000","last_became_active":"2023-08-11T15:44:19.858512+0000","last_became_peered":"2023-08-11T15:44:19.858512+0000","last_unstale":"2023-08-11T15:44:26.909786+0000","last_undegraded":"2023-08-11T15:44:26.909786+0000","last_fullsized":"2023-08-11T15:44:26.909786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:53:27.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190057+0000","last_change":"2023-08-11T15:44:19.864063+0000","last_active":"2023-08-11T15:44:54.190057+0000","last_peered":"2023-08-11T15:44:54.190057+0000","last_clean":"2023-08-11T15:44:54.190057+0000","last_became_active":"2023-08-11T15:44:19.863081+0000","last_became_peered":"2023-08-11T15:44:19.863081+0000","last_unstale":"2023-08-11T15:44:54.190057+0000","last_undegraded":"2023-08-11T15:44:54.190057+0000","last_fullsized":"2023-08-11T15:44:54.190057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:19:47.689166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196608+0000","last_change":"2023-08-11T15:44:19.855347+0000","last_active":"2023-08-11T15:44:25.196608+0000","last_peered":"2023-08-11T15:44:25.196608+0000","last_clean":"2023-08-11T15:44:25.196608+0000","last_became_active":"2023-08-11T15:44:19.855025+0000","last_became_peered":"2023-08-11T15:44:19.855025+0000","last_unstale":"2023-08-11T15:44:25.196608+0000","last_undegraded":"2023-08-11T15:44:25.196608+0000","last_fullsized":"2023-08-11T15:44:25.196608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:52:42.424929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398761+0000","last_change":"2023-08-11T15:44:19.862153+0000","last_active":"2023-08-11T15:44:25.398761+0000","last_peered":"2023-08-11T15:44:25.398761+0000","last_clean":"2023-08-11T15:44:25.398761+0000","last_became_active":"2023-08-11T15:44:19.861830+0000","last_became_peered":"2023-08-11T15:44:19.861830+0000","last_unstale":"2023-08-11T15:44:25.398761+0000","last_undegraded":"2023-08-11T15:44:25.398761+0000","last_fullsized":"2023-08-11T15:44:25.398761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:54:03.374483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240557+0000","last_change":"2023-08-11T15:44:19.857184+0000","last_active":"2023-08-11T15:44:26.240557+0000","last_peered":"2023-08-11T15:44:26.240557+0000","last_clean":"2023-08-11T15:44:26.240557+0000","last_became_active":"2023-08-11T15:44:19.856898+0000","last_became_peered":"2023-08-11T15:44:19.856898+0000","last_unstale":"2023-08-11T15:44:26.240557+0000","last_undegraded":"2023-08-11T15:44:26.240557+0000","last_fullsized":"2023-08-11T15:44:26.240557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:35:16.949556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240612+0000","last_change":"2023-08-11T15:44:19.852808+0000","last_active":"2023-08-11T15:44:26.240612+0000","last_peered":"2023-08-11T15:44:26.240612+0000","last_clean":"2023-08-11T15:44:26.240612+0000","last_became_active":"2023-08-11T15:44:19.852477+0000","last_became_peered":"2023-08-11T15:44:19.852477+0000","last_unstale":"2023-08-11T15:44:26.240612+0000","last_undegraded":"2023-08-11T15:44:26.240612+0000","last_fullsized":"2023-08-11T15:44:26.240612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:44:30.565373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496877+0000","last_change":"2023-08-11T15:44:19.856273+0000","last_active":"2023-08-11T15:44:27.496877+0000","last_peered":"2023-08-11T15:44:27.496877+0000","last_clean":"2023-08-11T15:44:27.496877+0000","last_became_active":"2023-08-11T15:44:19.854579+0000","last_became_peered":"2023-08-11T15:44:19.854579+0000","last_unstale":"2023-08-11T15:44:27.496877+0000","last_undegraded":"2023-08-11T15:44:27.496877+0000","last_fullsized":"2023-08-11T15:44:27.496877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:49:32.949731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184320+0000","last_change":"2023-08-11T15:44:19.866820+0000","last_active":"2023-08-11T15:44:25.184320+0000","last_peered":"2023-08-11T15:44:25.184320+0000","last_clean":"2023-08-11T15:44:25.184320+0000","last_became_active":"2023-08-11T15:44:19.866507+0000","last_became_peered":"2023-08-11T15:44:19.866507+0000","last_unstale":"2023-08-11T15:44:25.184320+0000","last_undegraded":"2023-08-11T15:44:25.184320+0000","last_fullsized":"2023-08-11T15:44:25.184320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:46.966328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181871+0000","last_change":"2023-08-11T15:44:19.865821+0000","last_active":"2023-08-11T15:44:25.181871+0000","last_peered":"2023-08-11T15:44:25.181871+0000","last_clean":"2023-08-11T15:44:25.181871+0000","last_became_active":"2023-08-11T15:44:19.865510+0000","last_became_peered":"2023-08-11T15:44:19.865510+0000","last_unstale":"2023-08-11T15:44:25.181871+0000","last_undegraded":"2023-08-11T15:44:25.181871+0000","last_fullsized":"2023-08-11T15:44:25.181871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:25:42.382893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130810+0000","last_change":"2023-08-11T15:44:19.863079+0000","last_active":"2023-08-11T15:44:27.130810+0000","last_peered":"2023-08-11T15:44:27.130810+0000","last_clean":"2023-08-11T15:44:27.130810+0000","last_became_active":"2023-08-11T15:44:19.862663+0000","last_became_peered":"2023-08-11T15:44:19.862663+0000","last_unstale":"2023-08-11T15:44:27.130810+0000","last_undegraded":"2023-08-11T15:44:27.130810+0000","last_fullsized":"2023-08-11T15:44:27.130810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:53.635387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130328+0000","last_change":"2023-08-11T15:44:24.201308+0000","last_active":"2023-08-11T15:44:27.130328+0000","last_peered":"2023-08-11T15:44:27.130328+0000","last_clean":"2023-08-11T15:44:27.130328+0000","last_became_active":"2023-08-11T15:44:24.200884+0000","last_became_peered":"2023-08-11T15:44:24.200884+0000","last_unstale":"2023-08-11T15:44:27.130328+0000","last_undegraded":"2023-08-11T15:44:27.130328+0000","last_fullsized":"2023-08-11T15:44:27.130328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:04:43.367558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181892+0000","last_change":"2023-08-11T15:44:19.155621+0000","last_active":"2023-08-11T15:44:25.181892+0000","last_peered":"2023-08-11T15:44:25.181892+0000","last_clean":"2023-08-11T15:44:25.181892+0000","last_became_active":"2023-08-11T15:44:19.154494+0000","last_became_peered":"2023-08-11T15:44:19.154494+0000","last_unstale":"2023-08-11T15:44:25.181892+0000","last_undegraded":"2023-08-11T15:44:25.181892+0000","last_fullsized":"2023-08-11T15:44:25.181892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:22:15.734115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179239+0000","last_change":"2023-08-11T15:44:19.853101+0000","last_active":"2023-08-11T15:44:25.179239+0000","last_peered":"2023-08-11T15:44:25.179239+0000","last_clean":"2023-08-11T15:44:25.179239+0000","last_became_active":"2023-08-11T15:44:19.852804+0000","last_became_peered":"2023-08-11T15:44:19.852804+0000","last_unstale":"2023-08-11T15:44:25.179239+0000","last_undegraded":"2023-08-11T15:44:25.179239+0000","last_fullsized":"2023-08-11T15:44:25.179239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:04:34.124031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196052+0000","last_change":"2023-08-11T15:44:24.198154+0000","last_active":"2023-08-11T15:44:25.196052+0000","last_peered":"2023-08-11T15:44:25.196052+0000","last_clean":"2023-08-11T15:44:25.196052+0000","last_became_active":"2023-08-11T15:44:24.197693+0000","last_became_peered":"2023-08-11T15:44:24.197693+0000","last_unstale":"2023-08-11T15:44:25.196052+0000","last_undegraded":"2023-08-11T15:44:25.196052+0000","last_fullsized":"2023-08-11T15:44:25.196052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:41:28.478746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179250+0000","last_change":"2023-08-11T15:44:19.147219+0000","last_active":"2023-08-11T15:44:25.179250+0000","last_peered":"2023-08-11T15:44:25.179250+0000","last_clean":"2023-08-11T15:44:25.179250+0000","last_became_active":"2023-08-11T15:44:19.146874+0000","last_became_peered":"2023-08-11T15:44:19.146874+0000","last_unstale":"2023-08-11T15:44:25.179250+0000","last_undegraded":"2023-08-11T15:44:25.179250+0000","last_fullsized":"2023-08-11T15:44:25.179250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:22:17.797682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241145+0000","last_change":"2023-08-11T15:44:19.854103+0000","last_active":"2023-08-11T15:44:26.241145+0000","last_peered":"2023-08-11T15:44:26.241145+0000","last_clean":"2023-08-11T15:44:26.241145+0000","last_became_active":"2023-08-11T15:44:19.853542+0000","last_became_peered":"2023-08-11T15:44:19.853542+0000","last_unstale":"2023-08-11T15:44:26.241145+0000","last_undegraded":"2023-08-11T15:44:26.241145+0000","last_fullsized":"2023-08-11T15:44:26.241145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:55:02.787594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196286+0000","last_change":"2023-08-11T15:44:24.202825+0000","last_active":"2023-08-11T15:44:25.196286+0000","last_peered":"2023-08-11T15:44:25.196286+0000","last_clean":"2023-08-11T15:44:25.196286+0000","last_became_active":"2023-08-11T15:44:24.202342+0000","last_became_peered":"2023-08-11T15:44:24.202342+0000","last_unstale":"2023-08-11T15:44:25.196286+0000","last_undegraded":"2023-08-11T15:44:25.196286+0000","last_fullsized":"2023-08-11T15:44:25.196286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:00.909726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196861+0000","last_change":"2023-08-11T15:44:19.148019+0000","last_active":"2023-08-11T15:44:54.196861+0000","last_peered":"2023-08-11T15:44:54.196861+0000","last_clean":"2023-08-11T15:44:54.196861+0000","last_became_active":"2023-08-11T15:44:19.147501+0000","last_became_peered":"2023-08-11T15:44:19.147501+0000","last_unstale":"2023-08-11T15:44:54.196861+0000","last_undegraded":"2023-08-11T15:44:54.196861+0000","last_fullsized":"2023-08-11T15:44:54.196861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:35.232097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497109+0000","last_change":"2023-08-11T15:44:19.865230+0000","last_active":"2023-08-11T15:44:27.497109+0000","last_peered":"2023-08-11T15:44:27.497109+0000","last_clean":"2023-08-11T15:44:27.497109+0000","last_became_active":"2023-08-11T15:44:19.864838+0000","last_became_peered":"2023-08-11T15:44:19.864838+0000","last_unstale":"2023-08-11T15:44:27.497109+0000","last_undegraded":"2023-08-11T15:44:27.497109+0000","last_fullsized":"2023-08-11T15:44:27.497109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:43:54.021675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196125+0000","last_change":"2023-08-11T15:44:24.192282+0000","last_active":"2023-08-11T15:44:25.196125+0000","last_peered":"2023-08-11T15:44:25.196125+0000","last_clean":"2023-08-11T15:44:25.196125+0000","last_became_active":"2023-08-11T15:44:24.191972+0000","last_became_peered":"2023-08-11T15:44:24.191972+0000","last_unstale":"2023-08-11T15:44:25.196125+0000","last_undegraded":"2023-08-11T15:44:25.196125+0000","last_fullsized":"2023-08-11T15:44:25.196125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:39.427988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196635+0000","last_change":"2023-08-11T15:44:19.147181+0000","last_active":"2023-08-11T15:44:54.196635+0000","last_peered":"2023-08-11T15:44:54.196635+0000","last_clean":"2023-08-11T15:44:54.196635+0000","last_became_active":"2023-08-11T15:44:19.146843+0000","last_became_peered":"2023-08-11T15:44:19.146843+0000","last_unstale":"2023-08-11T15:44:54.196635+0000","last_undegraded":"2023-08-11T15:44:54.196635+0000","last_fullsized":"2023-08-11T15:44:54.196635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:36:39.248789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043920+0000","last_change":"2023-08-11T15:44:19.858611+0000","last_active":"2023-08-11T15:44:27.043920+0000","last_peered":"2023-08-11T15:44:27.043920+0000","last_clean":"2023-08-11T15:44:27.043920+0000","last_became_active":"2023-08-11T15:44:19.858288+0000","last_became_peered":"2023-08-11T15:44:19.858288+0000","last_unstale":"2023-08-11T15:44:27.043920+0000","last_undegraded":"2023-08-11T15:44:27.043920+0000","last_fullsized":"2023-08-11T15:44:27.043920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:41.338062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130820+0000","last_change":"2023-08-11T15:44:24.192621+0000","last_active":"2023-08-11T15:44:27.130820+0000","last_peered":"2023-08-11T15:44:27.130820+0000","last_clean":"2023-08-11T15:44:27.130820+0000","last_became_active":"2023-08-11T15:44:24.192183+0000","last_became_peered":"2023-08-11T15:44:24.192183+0000","last_unstale":"2023-08-11T15:44:27.130820+0000","last_undegraded":"2023-08-11T15:44:27.130820+0000","last_fullsized":"2023-08-11T15:44:27.130820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:08:01.082842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043685+0000","last_change":"2023-08-11T15:44:19.149437+0000","last_active":"2023-08-11T15:44:27.043685+0000","last_peered":"2023-08-11T15:44:27.043685+0000","last_clean":"2023-08-11T15:44:27.043685+0000","last_became_active":"2023-08-11T15:44:19.149086+0000","last_became_peered":"2023-08-11T15:44:19.149086+0000","last_unstale":"2023-08-11T15:44:27.043685+0000","last_undegraded":"2023-08-11T15:44:27.043685+0000","last_fullsized":"2023-08-11T15:44:27.043685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:38.263428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196160+0000","last_change":"2023-08-11T15:44:19.865452+0000","last_active":"2023-08-11T15:44:54.196160+0000","last_peered":"2023-08-11T15:44:54.196160+0000","last_clean":"2023-08-11T15:44:54.196160+0000","last_became_active":"2023-08-11T15:44:19.865133+0000","last_became_peered":"2023-08-11T15:44:19.865133+0000","last_unstale":"2023-08-11T15:44:54.196160+0000","last_undegraded":"2023-08-11T15:44:54.196160+0000","last_fullsized":"2023-08-11T15:44:54.196160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:20.210619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130658+0000","last_change":"2023-08-11T15:44:24.201817+0000","last_active":"2023-08-11T15:44:27.130658+0000","last_peered":"2023-08-11T15:44:27.130658+0000","last_clean":"2023-08-11T15:44:27.130658+0000","last_became_active":"2023-08-11T15:44:24.200766+0000","last_became_peered":"2023-08-11T15:44:24.200766+0000","last_unstale":"2023-08-11T15:44:27.130658+0000","last_undegraded":"2023-08-11T15:44:27.130658+0000","last_fullsized":"2023-08-11T15:44:27.130658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:44:12.538791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909254+0000","last_change":"2023-08-11T15:44:19.151213+0000","last_active":"2023-08-11T15:44:26.909254+0000","last_peered":"2023-08-11T15:44:26.909254+0000","last_clean":"2023-08-11T15:44:26.909254+0000","last_became_active":"2023-08-11T15:44:19.150914+0000","last_became_peered":"2023-08-11T15:44:19.150914+0000","last_unstale":"2023-08-11T15:44:26.909254+0000","last_undegraded":"2023-08-11T15:44:26.909254+0000","last_fullsized":"2023-08-11T15:44:26.909254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:33:38.997413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043814+0000","last_change":"2023-08-11T15:44:19.858238+0000","last_active":"2023-08-11T15:44:27.043814+0000","last_peered":"2023-08-11T15:44:27.043814+0000","last_clean":"2023-08-11T15:44:27.043814+0000","last_became_active":"2023-08-11T15:44:19.857919+0000","last_became_peered":"2023-08-11T15:44:19.857919+0000","last_unstale":"2023-08-11T15:44:27.043814+0000","last_undegraded":"2023-08-11T15:44:27.043814+0000","last_fullsized":"2023-08-11T15:44:27.043814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:20:28.876449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184406+0000","last_change":"2023-08-11T15:44:24.201485+0000","last_active":"2023-08-11T15:44:25.184406+0000","last_peered":"2023-08-11T15:44:25.184406+0000","last_clean":"2023-08-11T15:44:25.184406+0000","last_became_active":"2023-08-11T15:44:24.200984+0000","last_became_peered":"2023-08-11T15:44:24.200984+0000","last_unstale":"2023-08-11T15:44:25.184406+0000","last_undegraded":"2023-08-11T15:44:25.184406+0000","last_fullsized":"2023-08-11T15:44:25.184406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:12.802792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398401+0000","last_change":"2023-08-11T15:44:19.149289+0000","last_active":"2023-08-11T15:44:25.398401+0000","last_peered":"2023-08-11T15:44:25.398401+0000","last_clean":"2023-08-11T15:44:25.398401+0000","last_became_active":"2023-08-11T15:44:19.148881+0000","last_became_peered":"2023-08-11T15:44:19.148881+0000","last_unstale":"2023-08-11T15:44:25.398401+0000","last_undegraded":"2023-08-11T15:44:25.398401+0000","last_fullsized":"2023-08-11T15:44:25.398401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:31.674718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178473+0000","last_change":"2023-08-11T15:44:19.867860+0000","last_active":"2023-08-11T15:44:25.178473+0000","last_peered":"2023-08-11T15:44:25.178473+0000","last_clean":"2023-08-11T15:44:25.178473+0000","last_became_active":"2023-08-11T15:44:19.867545+0000","last_became_peered":"2023-08-11T15:44:19.867545+0000","last_unstale":"2023-08-11T15:44:25.178473+0000","last_undegraded":"2023-08-11T15:44:25.178473+0000","last_fullsized":"2023-08-11T15:44:25.178473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:22.607885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241050+0000","last_change":"2023-08-11T15:44:24.197251+0000","last_active":"2023-08-11T15:44:26.241050+0000","last_peered":"2023-08-11T15:44:26.241050+0000","last_clean":"2023-08-11T15:44:26.241050+0000","last_became_active":"2023-08-11T15:44:24.196735+0000","last_became_peered":"2023-08-11T15:44:24.196735+0000","last_unstale":"2023-08-11T15:44:26.241050+0000","last_undegraded":"2023-08-11T15:44:26.241050+0000","last_fullsized":"2023-08-11T15:44:26.241050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:28:12.008317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043322+0000","last_change":"2023-08-11T15:44:19.146196+0000","last_active":"2023-08-11T15:44:27.043322+0000","last_peered":"2023-08-11T15:44:27.043322+0000","last_clean":"2023-08-11T15:44:27.043322+0000","last_became_active":"2023-08-11T15:44:19.145812+0000","last_became_peered":"2023-08-11T15:44:19.145812+0000","last_unstale":"2023-08-11T15:44:27.043322+0000","last_undegraded":"2023-08-11T15:44:27.043322+0000","last_fullsized":"2023-08-11T15:44:27.043322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:57.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181289+0000","last_change":"2023-08-11T15:44:19.855389+0000","last_active":"2023-08-11T15:44:25.181289+0000","last_peered":"2023-08-11T15:44:25.181289+0000","last_clean":"2023-08-11T15:44:25.181289+0000","last_became_active":"2023-08-11T15:44:19.855063+0000","last_became_peered":"2023-08-11T15:44:19.855063+0000","last_unstale":"2023-08-11T15:44:25.181289+0000","last_undegraded":"2023-08-11T15:44:25.181289+0000","last_fullsized":"2023-08-11T15:44:25.181289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:06:46.738170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178587+0000","last_change":"2023-08-11T15:44:24.192450+0000","last_active":"2023-08-11T15:44:25.178587+0000","last_peered":"2023-08-11T15:44:25.178587+0000","last_clean":"2023-08-11T15:44:25.178587+0000","last_became_active":"2023-08-11T15:44:24.191989+0000","last_became_peered":"2023-08-11T15:44:24.191989+0000","last_unstale":"2023-08-11T15:44:25.178587+0000","last_undegraded":"2023-08-11T15:44:25.178587+0000","last_fullsized":"2023-08-11T15:44:25.178587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:54:21.852954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909733+0000","last_change":"2023-08-11T15:44:19.149002+0000","last_active":"2023-08-11T15:44:26.909733+0000","last_peered":"2023-08-11T15:44:26.909733+0000","last_clean":"2023-08-11T15:44:26.909733+0000","last_became_active":"2023-08-11T15:44:19.148635+0000","last_became_peered":"2023-08-11T15:44:19.148635+0000","last_unstale":"2023-08-11T15:44:26.909733+0000","last_undegraded":"2023-08-11T15:44:26.909733+0000","last_fullsized":"2023-08-11T15:44:26.909733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:27:02.711671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497221+0000","last_change":"2023-08-11T15:44:19.858067+0000","last_active":"2023-08-11T15:44:27.497221+0000","last_peered":"2023-08-11T15:44:27.497221+0000","last_clean":"2023-08-11T15:44:27.497221+0000","last_became_active":"2023-08-11T15:44:19.856641+0000","last_became_peered":"2023-08-11T15:44:19.856641+0000","last_unstale":"2023-08-11T15:44:27.497221+0000","last_undegraded":"2023-08-11T15:44:27.497221+0000","last_fullsized":"2023-08-11T15:44:27.497221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:29.896919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181349+0000","last_change":"2023-08-11T15:44:24.196806+0000","last_active":"2023-08-11T15:44:25.181349+0000","last_peered":"2023-08-11T15:44:25.181349+0000","last_clean":"2023-08-11T15:44:25.181349+0000","last_became_active":"2023-08-11T15:44:24.195268+0000","last_became_peered":"2023-08-11T15:44:24.195268+0000","last_unstale":"2023-08-11T15:44:25.181349+0000","last_undegraded":"2023-08-11T15:44:25.181349+0000","last_fullsized":"2023-08-11T15:44:25.181349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:45:38.922930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.496972+0000","last_change":"2023-08-11T15:44:54.190469+0000","last_active":"2023-08-11T15:44:54.190469+0000","last_peered":"2023-08-11T15:44:27.496972+0000","last_clean":"2023-08-11T15:44:27.496972+0000","last_became_active":"2023-08-11T15:44:19.144155+0000","last_became_peered":"2023-08-11T15:44:19.144155+0000","last_unstale":"2023-08-11T15:44:27.496972+0000","last_undegraded":"2023-08-11T15:44:27.496972+0000","last_fullsized":"2023-08-11T15:44:27.496972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240742+0000","last_change":"2023-08-11T15:44:19.853881+0000","last_active":"2023-08-11T15:44:26.240742+0000","last_peered":"2023-08-11T15:44:26.240742+0000","last_clean":"2023-08-11T15:44:26.240742+0000","last_became_active":"2023-08-11T15:44:19.853139+0000","last_became_peered":"2023-08-11T15:44:19.853139+0000","last_unstale":"2023-08-11T15:44:26.240742+0000","last_undegraded":"2023-08-11T15:44:26.240742+0000","last_fullsized":"2023-08-11T15:44:26.240742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:05.315621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196399+0000","last_change":"2023-08-11T15:44:24.201620+0000","last_active":"2023-08-11T15:44:25.196399+0000","last_peered":"2023-08-11T15:44:25.196399+0000","last_clean":"2023-08-11T15:44:25.196399+0000","last_became_active":"2023-08-11T15:44:24.201162+0000","last_became_peered":"2023-08-11T15:44:24.201162+0000","last_unstale":"2023-08-11T15:44:25.196399+0000","last_undegraded":"2023-08-11T15:44:25.196399+0000","last_fullsized":"2023-08-11T15:44:25.196399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:54:06.512139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043904+0000","last_change":"2023-08-11T15:44:19.156899+0000","last_active":"2023-08-11T15:44:27.043904+0000","last_peered":"2023-08-11T15:44:27.043904+0000","last_clean":"2023-08-11T15:44:27.043904+0000","last_became_active":"2023-08-11T15:44:19.156643+0000","last_became_peered":"2023-08-11T15:44:19.156643+0000","last_unstale":"2023-08-11T15:44:27.043904+0000","last_undegraded":"2023-08-11T15:44:27.043904+0000","last_fullsized":"2023-08-11T15:44:27.043904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:27.691696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497337+0000","last_change":"2023-08-11T15:44:19.857828+0000","last_active":"2023-08-11T15:44:27.497337+0000","last_peered":"2023-08-11T15:44:27.497337+0000","last_clean":"2023-08-11T15:44:27.497337+0000","last_became_active":"2023-08-11T15:44:19.856781+0000","last_became_peered":"2023-08-11T15:44:19.856781+0000","last_unstale":"2023-08-11T15:44:27.497337+0000","last_undegraded":"2023-08-11T15:44:27.497337+0000","last_fullsized":"2023-08-11T15:44:27.497337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:59.861717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181369+0000","last_change":"2023-08-11T15:44:24.193213+0000","last_active":"2023-08-11T15:44:25.181369+0000","last_peered":"2023-08-11T15:44:25.181369+0000","last_clean":"2023-08-11T15:44:25.181369+0000","last_became_active":"2023-08-11T15:44:24.192819+0000","last_became_peered":"2023-08-11T15:44:24.192819+0000","last_unstale":"2023-08-11T15:44:25.181369+0000","last_undegraded":"2023-08-11T15:44:25.181369+0000","last_fullsized":"2023-08-11T15:44:25.181369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:16.938740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'10","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:52.186980+0000","last_change":"2023-08-11T15:44:19.149753+0000","last_active":"2023-08-11T15:44:52.186980+0000","last_peered":"2023-08-11T15:44:52.186980+0000","last_clean":"2023-08-11T15:44:52.186980+0000","last_became_active":"2023-08-11T15:44:19.149420+0000","last_became_peered":"2023-08-11T15:44:19.149420+0000","last_unstale":"2023-08-11T15:44:52.186980+0000","last_undegraded":"2023-08-11T15:44:52.186980+0000","last_fullsized":"2023-08-11T15:44:52.186980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:58.376378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196006+0000","last_change":"2023-08-11T15:44:19.861342+0000","last_active":"2023-08-11T15:44:54.196006+0000","last_peered":"2023-08-11T15:44:54.196006+0000","last_clean":"2023-08-11T15:44:54.196006+0000","last_became_active":"2023-08-11T15:44:19.860978+0000","last_became_peered":"2023-08-11T15:44:19.860978+0000","last_unstale":"2023-08-11T15:44:54.196006+0000","last_undegraded":"2023-08-11T15:44:54.196006+0000","last_fullsized":"2023-08-11T15:44:54.196006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:52.637866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044274+0000","last_change":"2023-08-11T15:44:24.191504+0000","last_active":"2023-08-11T15:44:27.044274+0000","last_peered":"2023-08-11T15:44:27.044274+0000","last_clean":"2023-08-11T15:44:27.044274+0000","last_became_active":"2023-08-11T15:44:24.191127+0000","last_became_peered":"2023-08-11T15:44:24.191127+0000","last_unstale":"2023-08-11T15:44:27.044274+0000","last_undegraded":"2023-08-11T15:44:27.044274+0000","last_fullsized":"2023-08-11T15:44:27.044274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:11.970278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130897+0000","last_change":"2023-08-11T15:44:19.148354+0000","last_active":"2023-08-11T15:44:27.130897+0000","last_peered":"2023-08-11T15:44:27.130897+0000","last_clean":"2023-08-11T15:44:27.130897+0000","last_became_active":"2023-08-11T15:44:19.148036+0000","last_became_peered":"2023-08-11T15:44:19.148036+0000","last_unstale":"2023-08-11T15:44:27.130897+0000","last_undegraded":"2023-08-11T15:44:27.130897+0000","last_fullsized":"2023-08-11T15:44:27.130897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:37:33.516998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398521+0000","last_change":"2023-08-11T15:44:19.862409+0000","last_active":"2023-08-11T15:44:25.398521+0000","last_peered":"2023-08-11T15:44:25.398521+0000","last_clean":"2023-08-11T15:44:25.398521+0000","last_became_active":"2023-08-11T15:44:19.861359+0000","last_became_peered":"2023-08-11T15:44:19.861359+0000","last_unstale":"2023-08-11T15:44:25.398521+0000","last_undegraded":"2023-08-11T15:44:25.398521+0000","last_fullsized":"2023-08-11T15:44:25.398521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:41.755533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184424+0000","last_change":"2023-08-11T15:44:24.197634+0000","last_active":"2023-08-11T15:44:25.184424+0000","last_peered":"2023-08-11T15:44:25.184424+0000","last_clean":"2023-08-11T15:44:25.184424+0000","last_became_active":"2023-08-11T15:44:24.197181+0000","last_became_peered":"2023-08-11T15:44:24.197181+0000","last_unstale":"2023-08-11T15:44:25.184424+0000","last_undegraded":"2023-08-11T15:44:25.184424+0000","last_fullsized":"2023-08-11T15:44:25.184424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:23:45.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043675+0000","last_change":"2023-08-11T15:44:19.147550+0000","last_active":"2023-08-11T15:44:27.043675+0000","last_peered":"2023-08-11T15:44:27.043675+0000","last_clean":"2023-08-11T15:44:27.043675+0000","last_became_active":"2023-08-11T15:44:19.147299+0000","last_became_peered":"2023-08-11T15:44:19.147299+0000","last_unstale":"2023-08-11T15:44:27.043675+0000","last_undegraded":"2023-08-11T15:44:27.043675+0000","last_fullsized":"2023-08-11T15:44:27.043675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:56.050406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189968+0000","last_change":"2023-08-11T15:44:19.862664+0000","last_active":"2023-08-11T15:44:54.189968+0000","last_peered":"2023-08-11T15:44:54.189968+0000","last_clean":"2023-08-11T15:44:54.189968+0000","last_became_active":"2023-08-11T15:44:19.862130+0000","last_became_peered":"2023-08-11T15:44:19.862130+0000","last_unstale":"2023-08-11T15:44:54.189968+0000","last_undegraded":"2023-08-11T15:44:54.189968+0000","last_fullsized":"2023-08-11T15:44:54.189968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:02.591318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181786+0000","last_change":"2023-08-11T15:44:24.199851+0000","last_active":"2023-08-11T15:44:25.181786+0000","last_peered":"2023-08-11T15:44:25.181786+0000","last_clean":"2023-08-11T15:44:25.181786+0000","last_became_active":"2023-08-11T15:44:24.199420+0000","last_became_peered":"2023-08-11T15:44:24.199420+0000","last_unstale":"2023-08-11T15:44:25.181786+0000","last_undegraded":"2023-08-11T15:44:25.181786+0000","last_fullsized":"2023-08-11T15:44:25.181786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:36:31.272229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196088+0000","last_change":"2023-08-11T15:44:19.145888+0000","last_active":"2023-08-11T15:44:25.196088+0000","last_peered":"2023-08-11T15:44:25.196088+0000","last_clean":"2023-08-11T15:44:25.196088+0000","last_became_active":"2023-08-11T15:44:19.145459+0000","last_became_peered":"2023-08-11T15:44:19.145459+0000","last_unstale":"2023-08-11T15:44:25.196088+0000","last_undegraded":"2023-08-11T15:44:25.196088+0000","last_fullsized":"2023-08-11T15:44:25.196088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:36.387374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189722+0000","last_change":"2023-08-11T15:44:19.860035+0000","last_active":"2023-08-11T15:44:54.189722+0000","last_peered":"2023-08-11T15:44:54.189722+0000","last_clean":"2023-08-11T15:44:54.189722+0000","last_became_active":"2023-08-11T15:44:19.859696+0000","last_became_peered":"2023-08-11T15:44:19.859696+0000","last_unstale":"2023-08-11T15:44:54.189722+0000","last_undegraded":"2023-08-11T15:44:54.189722+0000","last_fullsized":"2023-08-11T15:44:54.189722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:24.878953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399086+0000","last_change":"2023-08-11T15:44:24.198255+0000","last_active":"2023-08-11T15:44:25.399086+0000","last_peered":"2023-08-11T15:44:25.399086+0000","last_clean":"2023-08-11T15:44:25.399086+0000","last_became_active":"2023-08-11T15:44:24.197825+0000","last_became_peered":"2023-08-11T15:44:24.197825+0000","last_unstale":"2023-08-11T15:44:25.399086+0000","last_undegraded":"2023-08-11T15:44:25.399086+0000","last_fullsized":"2023-08-11T15:44:25.399086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:41:49.976978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496775+0000","last_change":"2023-08-11T15:44:19.151410+0000","last_active":"2023-08-11T15:44:27.496775+0000","last_peered":"2023-08-11T15:44:27.496775+0000","last_clean":"2023-08-11T15:44:27.496775+0000","last_became_active":"2023-08-11T15:44:19.151117+0000","last_became_peered":"2023-08-11T15:44:19.151117+0000","last_unstale":"2023-08-11T15:44:27.496775+0000","last_undegraded":"2023-08-11T15:44:27.496775+0000","last_fullsized":"2023-08-11T15:44:27.496775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:11.127495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043467+0000","last_change":"2023-08-11T15:44:19.853273+0000","last_active":"2023-08-11T15:44:27.043467+0000","last_peered":"2023-08-11T15:44:27.043467+0000","last_clean":"2023-08-11T15:44:27.043467+0000","last_became_active":"2023-08-11T15:44:19.852914+0000","last_became_peered":"2023-08-11T15:44:19.852914+0000","last_unstale":"2023-08-11T15:44:27.043467+0000","last_undegraded":"2023-08-11T15:44:27.043467+0000","last_fullsized":"2023-08-11T15:44:27.043467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:45.110625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043436+0000","last_change":"2023-08-11T15:44:24.194429+0000","last_active":"2023-08-11T15:44:27.043436+0000","last_peered":"2023-08-11T15:44:27.043436+0000","last_clean":"2023-08-11T15:44:27.043436+0000","last_became_active":"2023-08-11T15:44:24.194089+0000","last_became_peered":"2023-08-11T15:44:24.194089+0000","last_unstale":"2023-08-11T15:44:27.043436+0000","last_undegraded":"2023-08-11T15:44:27.043436+0000","last_fullsized":"2023-08-11T15:44:27.043436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:53.530309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194680+0000","last_change":"2023-08-11T15:44:19.151072+0000","last_active":"2023-08-11T15:44:54.194680+0000","last_peered":"2023-08-11T15:44:54.194680+0000","last_clean":"2023-08-11T15:44:54.194680+0000","last_became_active":"2023-08-11T15:44:19.147713+0000","last_became_peered":"2023-08-11T15:44:19.147713+0000","last_unstale":"2023-08-11T15:44:54.194680+0000","last_undegraded":"2023-08-11T15:44:54.194680+0000","last_fullsized":"2023-08-11T15:44:54.194680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:09:42.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184182+0000","last_change":"2023-08-11T15:44:19.862259+0000","last_active":"2023-08-11T15:44:25.184182+0000","last_peered":"2023-08-11T15:44:25.184182+0000","last_clean":"2023-08-11T15:44:25.184182+0000","last_became_active":"2023-08-11T15:44:19.861910+0000","last_became_peered":"2023-08-11T15:44:19.861910+0000","last_unstale":"2023-08-11T15:44:25.184182+0000","last_undegraded":"2023-08-11T15:44:25.184182+0000","last_fullsized":"2023-08-11T15:44:25.184182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:06:41.610675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.497633+0000","last_change":"2023-08-11T15:44:54.195412+0000","last_active":"2023-08-11T15:44:54.195412+0000","last_peered":"2023-08-11T15:44:27.497633+0000","last_clean":"2023-08-11T15:44:27.497633+0000","last_became_active":"2023-08-11T15:44:24.300268+0000","last_became_peered":"2023-08-11T15:44:24.300268+0000","last_unstale":"2023-08-11T15:44:27.497633+0000","last_undegraded":"2023-08-11T15:44:27.497633+0000","last_fullsized":"2023-08-11T15:44:27.497633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181579+0000","last_change":"2023-08-11T15:44:19.152608+0000","last_active":"2023-08-11T15:44:25.181579+0000","last_peered":"2023-08-11T15:44:25.181579+0000","last_clean":"2023-08-11T15:44:25.181579+0000","last_became_active":"2023-08-11T15:44:19.152255+0000","last_became_peered":"2023-08-11T15:44:19.152255+0000","last_unstale":"2023-08-11T15:44:25.181579+0000","last_undegraded":"2023-08-11T15:44:25.181579+0000","last_fullsized":"2023-08-11T15:44:25.181579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:33.299760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909653+0000","last_change":"2023-08-11T15:44:19.864003+0000","last_active":"2023-08-11T15:44:26.909653+0000","last_peered":"2023-08-11T15:44:26.909653+0000","last_clean":"2023-08-11T15:44:26.909653+0000","last_became_active":"2023-08-11T15:44:19.863661+0000","last_became_peered":"2023-08-11T15:44:19.863661+0000","last_unstale":"2023-08-11T15:44:26.909653+0000","last_undegraded":"2023-08-11T15:44:26.909653+0000","last_fullsized":"2023-08-11T15:44:26.909653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:20:35.118164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398869+0000","last_change":"2023-08-11T15:44:24.194123+0000","last_active":"2023-08-11T15:44:25.398869+0000","last_peered":"2023-08-11T15:44:25.398869+0000","last_clean":"2023-08-11T15:44:25.398869+0000","last_became_active":"2023-08-11T15:44:24.193729+0000","last_became_peered":"2023-08-11T15:44:24.193729+0000","last_unstale":"2023-08-11T15:44:25.398869+0000","last_undegraded":"2023-08-11T15:44:25.398869+0000","last_fullsized":"2023-08-11T15:44:25.398869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:11:10.722179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189631+0000","last_change":"2023-08-11T15:44:19.151367+0000","last_active":"2023-08-11T15:44:54.189631+0000","last_peered":"2023-08-11T15:44:54.189631+0000","last_clean":"2023-08-11T15:44:54.189631+0000","last_became_active":"2023-08-11T15:44:19.150991+0000","last_became_peered":"2023-08-11T15:44:19.150991+0000","last_unstale":"2023-08-11T15:44:54.189631+0000","last_undegraded":"2023-08-11T15:44:54.189631+0000","last_fullsized":"2023-08-11T15:44:54.189631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:03:03.249036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.909318+0000","last_change":"2023-08-11T15:44:19.858579+0000","last_active":"2023-08-11T15:44:26.909318+0000","last_peered":"2023-08-11T15:44:26.909318+0000","last_clean":"2023-08-11T15:44:26.909318+0000","last_became_active":"2023-08-11T15:44:19.858184+0000","last_became_peered":"2023-08-11T15:44:19.858184+0000","last_unstale":"2023-08-11T15:44:26.909318+0000","last_undegraded":"2023-08-11T15:44:26.909318+0000","last_fullsized":"2023-08-11T15:44:26.909318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:19:38.034222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399074+0000","last_change":"2023-08-11T15:44:24.300526+0000","last_active":"2023-08-11T15:44:25.399074+0000","last_peered":"2023-08-11T15:44:25.399074+0000","last_clean":"2023-08-11T15:44:25.399074+0000","last_became_active":"2023-08-11T15:44:24.299883+0000","last_became_peered":"2023-08-11T15:44:24.299883+0000","last_unstale":"2023-08-11T15:44:25.399074+0000","last_undegraded":"2023-08-11T15:44:25.399074+0000","last_fullsized":"2023-08-11T15:44:25.399074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:53:45.487528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043468+0000","last_change":"2023-08-11T15:44:19.858116+0000","last_active":"2023-08-11T15:44:27.043468+0000","last_peered":"2023-08-11T15:44:27.043468+0000","last_clean":"2023-08-11T15:44:27.043468+0000","last_became_active":"2023-08-11T15:44:19.857728+0000","last_became_peered":"2023-08-11T15:44:19.857728+0000","last_unstale":"2023-08-11T15:44:27.043468+0000","last_undegraded":"2023-08-11T15:44:27.043468+0000","last_fullsized":"2023-08-11T15:44:27.043468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:56:36.953948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398236+0000","last_change":"2023-08-11T15:44:19.157904+0000","last_active":"2023-08-11T15:44:25.398236+0000","last_peered":"2023-08-11T15:44:25.398236+0000","last_clean":"2023-08-11T15:44:25.398236+0000","last_became_active":"2023-08-11T15:44:19.157497+0000","last_became_peered":"2023-08-11T15:44:19.157497+0000","last_unstale":"2023-08-11T15:44:25.398236+0000","last_undegraded":"2023-08-11T15:44:25.398236+0000","last_fullsized":"2023-08-11T15:44:25.398236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:10.337630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189839+0000","last_change":"2023-08-11T15:44:24.201246+0000","last_active":"2023-08-11T15:44:54.189839+0000","last_peered":"2023-08-11T15:44:54.189839+0000","last_clean":"2023-08-11T15:44:54.189839+0000","last_became_active":"2023-08-11T15:44:24.200810+0000","last_became_peered":"2023-08-11T15:44:24.200810+0000","last_unstale":"2023-08-11T15:44:54.189839+0000","last_undegraded":"2023-08-11T15:44:54.189839+0000","last_fullsized":"2023-08-11T15:44:54.189839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:47:27.192243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178569+0000","last_change":"2023-08-11T15:44:19.846296+0000","last_active":"2023-08-11T15:44:25.178569+0000","last_peered":"2023-08-11T15:44:25.178569+0000","last_clean":"2023-08-11T15:44:25.178569+0000","last_became_active":"2023-08-11T15:44:19.845890+0000","last_became_peered":"2023-08-11T15:44:19.845890+0000","last_unstale":"2023-08-11T15:44:25.178569+0000","last_undegraded":"2023-08-11T15:44:25.178569+0000","last_fullsized":"2023-08-11T15:44:25.178569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:44:22.834705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196213+0000","last_change":"2023-08-11T15:44:19.144202+0000","last_active":"2023-08-11T15:44:25.196213+0000","last_peered":"2023-08-11T15:44:25.196213+0000","last_clean":"2023-08-11T15:44:25.196213+0000","last_became_active":"2023-08-11T15:44:19.143789+0000","last_became_peered":"2023-08-11T15:44:19.143789+0000","last_unstale":"2023-08-11T15:44:25.196213+0000","last_undegraded":"2023-08-11T15:44:25.196213+0000","last_fullsized":"2023-08-11T15:44:25.196213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:57:56.919463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189289+0000","last_change":"2023-08-11T15:43:14.593634+0000","last_active":"2023-08-11T15:44:54.189289+0000","last_peered":"2023-08-11T15:44:54.189289+0000","last_clean":"2023-08-11T15:44:54.189289+0000","last_became_active":"2023-08-11T15:43:14.593164+0000","last_became_peered":"2023-08-11T15:43:14.593164+0000","last_unstale":"2023-08-11T15:44:54.189289+0000","last_undegraded":"2023-08-11T15:44:54.189289+0000","last_fullsized":"2023-08-11T15:44:54.189289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_clean_scrub_stamp":"2023-08-11T15:40:09.009169+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:15.991425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044055+0000","last_change":"2023-08-11T15:44:24.300703+0000","last_active":"2023-08-11T15:44:27.044055+0000","last_peered":"2023-08-11T15:44:27.044055+0000","last_clean":"2023-08-11T15:44:27.044055+0000","last_became_active":"2023-08-11T15:44:24.300309+0000","last_became_peered":"2023-08-11T15:44:24.300309+0000","last_unstale":"2023-08-11T15:44:27.044055+0000","last_undegraded":"2023-08-11T15:44:27.044055+0000","last_fullsized":"2023-08-11T15:44:27.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:41.018379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398223+0000","last_change":"2023-08-11T15:44:19.858934+0000","last_active":"2023-08-11T15:44:25.398223+0000","last_peered":"2023-08-11T15:44:25.398223+0000","last_clean":"2023-08-11T15:44:25.398223+0000","last_became_active":"2023-08-11T15:44:19.858613+0000","last_became_peered":"2023-08-11T15:44:19.858613+0000","last_unstale":"2023-08-11T15:44:25.398223+0000","last_undegraded":"2023-08-11T15:44:25.398223+0000","last_fullsized":"2023-08-11T15:44:25.398223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:07.938228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189544+0000","last_change":"2023-08-11T15:44:19.148113+0000","last_active":"2023-08-11T15:44:54.189544+0000","last_peered":"2023-08-11T15:44:54.189544+0000","last_clean":"2023-08-11T15:44:54.189544+0000","last_became_active":"2023-08-11T15:44:19.147793+0000","last_became_peered":"2023-08-11T15:44:19.147793+0000","last_unstale":"2023-08-11T15:44:54.189544+0000","last_undegraded":"2023-08-11T15:44:54.189544+0000","last_fullsized":"2023-08-11T15:44:54.189544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:07:38.923167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184402+0000","last_change":"2023-08-11T15:44:24.194418+0000","last_active":"2023-08-11T15:44:25.184402+0000","last_peered":"2023-08-11T15:44:25.184402+0000","last_clean":"2023-08-11T15:44:25.184402+0000","last_became_active":"2023-08-11T15:44:24.193943+0000","last_became_peered":"2023-08-11T15:44:24.193943+0000","last_unstale":"2023-08-11T15:44:25.184402+0000","last_undegraded":"2023-08-11T15:44:25.184402+0000","last_fullsized":"2023-08-11T15:44:25.184402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:35.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.196145+0000","last_change":"2023-08-11T15:44:19.151476+0000","last_active":"2023-08-11T15:44:54.196145+0000","last_peered":"2023-08-11T15:44:54.196145+0000","last_clean":"2023-08-11T15:44:54.196145+0000","last_became_active":"2023-08-11T15:44:19.151140+0000","last_became_peered":"2023-08-11T15:44:19.151140+0000","last_unstale":"2023-08-11T15:44:54.196145+0000","last_undegraded":"2023-08-11T15:44:54.196145+0000","last_fullsized":"2023-08-11T15:44:54.196145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:08:09.404210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398895+0000","last_change":"2023-08-11T15:44:19.862255+0000","last_active":"2023-08-11T15:44:25.398895+0000","last_peered":"2023-08-11T15:44:25.398895+0000","last_clean":"2023-08-11T15:44:25.398895+0000","last_became_active":"2023-08-11T15:44:19.861907+0000","last_became_peered":"2023-08-11T15:44:19.861907+0000","last_unstale":"2023-08-11T15:44:25.398895+0000","last_undegraded":"2023-08-11T15:44:25.398895+0000","last_fullsized":"2023-08-11T15:44:25.398895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:03.436493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178469+0000","last_change":"2023-08-11T15:44:24.300513+0000","last_active":"2023-08-11T15:44:25.178469+0000","last_peered":"2023-08-11T15:44:25.178469+0000","last_clean":"2023-08-11T15:44:25.178469+0000","last_became_active":"2023-08-11T15:44:24.300074+0000","last_became_peered":"2023-08-11T15:44:24.300074+0000","last_unstale":"2023-08-11T15:44:25.178469+0000","last_undegraded":"2023-08-11T15:44:25.178469+0000","last_fullsized":"2023-08-11T15:44:25.178469+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:16:21.212112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181835+0000","last_change":"2023-08-11T15:44:19.153794+0000","last_active":"2023-08-11T15:44:25.181835+0000","last_peered":"2023-08-11T15:44:25.181835+0000","last_clean":"2023-08-11T15:44:25.181835+0000","last_became_active":"2023-08-11T15:44:19.153392+0000","last_became_peered":"2023-08-11T15:44:19.153392+0000","last_unstale":"2023-08-11T15:44:25.181835+0000","last_undegraded":"2023-08-11T15:44:25.181835+0000","last_fullsized":"2023-08-11T15:44:25.181835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:59:47.991902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179190+0000","last_change":"2023-08-11T15:44:19.859799+0000","last_active":"2023-08-11T15:44:25.179190+0000","last_peered":"2023-08-11T15:44:25.179190+0000","last_clean":"2023-08-11T15:44:25.179190+0000","last_became_active":"2023-08-11T15:44:19.859411+0000","last_became_peered":"2023-08-11T15:44:19.859411+0000","last_unstale":"2023-08-11T15:44:25.179190+0000","last_undegraded":"2023-08-11T15:44:25.179190+0000","last_fullsized":"2023-08-11T15:44:25.179190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:19.443644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130327+0000","last_change":"2023-08-11T15:44:24.190272+0000","last_active":"2023-08-11T15:44:27.130327+0000","last_peered":"2023-08-11T15:44:27.130327+0000","last_clean":"2023-08-11T15:44:27.130327+0000","last_became_active":"2023-08-11T15:44:24.189849+0000","last_became_peered":"2023-08-11T15:44:24.189849+0000","last_unstale":"2023-08-11T15:44:27.130327+0000","last_undegraded":"2023-08-11T15:44:27.130327+0000","last_fullsized":"2023-08-11T15:44:27.130327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:26:26.603082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196757+0000","last_change":"2023-08-11T15:44:19.154985+0000","last_active":"2023-08-11T15:44:25.196757+0000","last_peered":"2023-08-11T15:44:25.196757+0000","last_clean":"2023-08-11T15:44:25.196757+0000","last_became_active":"2023-08-11T15:44:19.154694+0000","last_became_peered":"2023-08-11T15:44:19.154694+0000","last_unstale":"2023-08-11T15:44:25.196757+0000","last_undegraded":"2023-08-11T15:44:25.196757+0000","last_fullsized":"2023-08-11T15:44:25.196757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:15.737671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241127+0000","last_change":"2023-08-11T15:44:19.852247+0000","last_active":"2023-08-11T15:44:26.241127+0000","last_peered":"2023-08-11T15:44:26.241127+0000","last_clean":"2023-08-11T15:44:26.241127+0000","last_became_active":"2023-08-11T15:44:19.851456+0000","last_became_peered":"2023-08-11T15:44:19.851456+0000","last_unstale":"2023-08-11T15:44:26.241127+0000","last_undegraded":"2023-08-11T15:44:26.241127+0000","last_fullsized":"2023-08-11T15:44:26.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:57:45.705759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196509+0000","last_change":"2023-08-11T15:44:24.300440+0000","last_active":"2023-08-11T15:44:25.196509+0000","last_peered":"2023-08-11T15:44:25.196509+0000","last_clean":"2023-08-11T15:44:25.196509+0000","last_became_active":"2023-08-11T15:44:24.299981+0000","last_became_peered":"2023-08-11T15:44:24.299981+0000","last_unstale":"2023-08-11T15:44:25.196509+0000","last_undegraded":"2023-08-11T15:44:25.196509+0000","last_fullsized":"2023-08-11T15:44:25.196509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:11:24.190521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.496438+0000","last_change":"2023-08-11T15:44:19.151015+0000","last_active":"2023-08-11T15:44:27.496438+0000","last_peered":"2023-08-11T15:44:27.496438+0000","last_clean":"2023-08-11T15:44:27.496438+0000","last_became_active":"2023-08-11T15:44:19.150677+0000","last_became_peered":"2023-08-11T15:44:19.150677+0000","last_unstale":"2023-08-11T15:44:27.496438+0000","last_undegraded":"2023-08-11T15:44:27.496438+0000","last_fullsized":"2023-08-11T15:44:27.496438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:38:34.368253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.195188+0000","last_change":"2023-08-11T15:44:19.861033+0000","last_active":"2023-08-11T15:44:54.195188+0000","last_peered":"2023-08-11T15:44:54.195188+0000","last_clean":"2023-08-11T15:44:54.195188+0000","last_became_active":"2023-08-11T15:44:19.860643+0000","last_became_peered":"2023-08-11T15:44:19.860643+0000","last_unstale":"2023-08-11T15:44:54.195188+0000","last_undegraded":"2023-08-11T15:44:54.195188+0000","last_fullsized":"2023-08-11T15:44:54.195188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:56:51.466526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.181930+0000","last_change":"2023-08-11T15:44:24.199389+0000","last_active":"2023-08-11T15:44:25.181930+0000","last_peered":"2023-08-11T15:44:25.181930+0000","last_clean":"2023-08-11T15:44:25.181930+0000","last_became_active":"2023-08-11T15:44:24.196226+0000","last_became_peered":"2023-08-11T15:44:24.196226+0000","last_unstale":"2023-08-11T15:44:25.181930+0000","last_undegraded":"2023-08-11T15:44:25.181930+0000","last_fullsized":"2023-08-11T15:44:25.181930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:03.253768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178850+0000","last_change":"2023-08-11T15:44:19.152895+0000","last_active":"2023-08-11T15:44:25.178850+0000","last_peered":"2023-08-11T15:44:25.178850+0000","last_clean":"2023-08-11T15:44:25.178850+0000","last_became_active":"2023-08-11T15:44:19.152577+0000","last_became_peered":"2023-08-11T15:44:19.152577+0000","last_unstale":"2023-08-11T15:44:25.178850+0000","last_undegraded":"2023-08-11T15:44:25.178850+0000","last_fullsized":"2023-08-11T15:44:25.178850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:41:12.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183940+0000","last_change":"2023-08-11T15:44:19.861101+0000","last_active":"2023-08-11T15:44:25.183940+0000","last_peered":"2023-08-11T15:44:25.183940+0000","last_clean":"2023-08-11T15:44:25.183940+0000","last_became_active":"2023-08-11T15:44:19.860529+0000","last_became_peered":"2023-08-11T15:44:19.860529+0000","last_unstale":"2023-08-11T15:44:25.183940+0000","last_undegraded":"2023-08-11T15:44:25.183940+0000","last_fullsized":"2023-08-11T15:44:25.183940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:07:25.463560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196480+0000","last_change":"2023-08-11T15:44:24.202856+0000","last_active":"2023-08-11T15:44:25.196480+0000","last_peered":"2023-08-11T15:44:25.196480+0000","last_clean":"2023-08-11T15:44:25.196480+0000","last_became_active":"2023-08-11T15:44:24.202317+0000","last_became_peered":"2023-08-11T15:44:24.202317+0000","last_unstale":"2023-08-11T15:44:25.196480+0000","last_undegraded":"2023-08-11T15:44:25.196480+0000","last_fullsized":"2023-08-11T15:44:25.196480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:12:44.969687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130438+0000","last_change":"2023-08-11T15:44:19.148711+0000","last_active":"2023-08-11T15:44:27.130438+0000","last_peered":"2023-08-11T15:44:27.130438+0000","last_clean":"2023-08-11T15:44:27.130438+0000","last_became_active":"2023-08-11T15:44:19.148361+0000","last_became_peered":"2023-08-11T15:44:19.148361+0000","last_unstale":"2023-08-11T15:44:27.130438+0000","last_undegraded":"2023-08-11T15:44:27.130438+0000","last_fullsized":"2023-08-11T15:44:27.130438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:45.703207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178814+0000","last_change":"2023-08-11T15:44:19.859562+0000","last_active":"2023-08-11T15:44:25.178814+0000","last_peered":"2023-08-11T15:44:25.178814+0000","last_clean":"2023-08-11T15:44:25.178814+0000","last_became_active":"2023-08-11T15:44:19.859243+0000","last_became_peered":"2023-08-11T15:44:19.859243+0000","last_unstale":"2023-08-11T15:44:25.178814+0000","last_undegraded":"2023-08-11T15:44:25.178814+0000","last_fullsized":"2023-08-11T15:44:25.178814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:27:50.690599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.196555+0000","last_change":"2023-08-11T15:44:24.300317+0000","last_active":"2023-08-11T15:44:25.196555+0000","last_peered":"2023-08-11T15:44:25.196555+0000","last_clean":"2023-08-11T15:44:25.196555+0000","last_became_active":"2023-08-11T15:44:24.299841+0000","last_became_peered":"2023-08-11T15:44:24.299841+0000","last_unstale":"2023-08-11T15:44:25.196555+0000","last_undegraded":"2023-08-11T15:44:25.196555+0000","last_fullsized":"2023-08-11T15:44:25.196555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:16.800305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.194494+0000","last_change":"2023-08-11T15:44:19.149111+0000","last_active":"2023-08-11T15:44:54.194494+0000","last_peered":"2023-08-11T15:44:54.194494+0000","last_clean":"2023-08-11T15:44:54.194494+0000","last_became_active":"2023-08-11T15:44:19.148173+0000","last_became_peered":"2023-08-11T15:44:19.148173+0000","last_unstale":"2023-08-11T15:44:54.194494+0000","last_undegraded":"2023-08-11T15:44:54.194494+0000","last_fullsized":"2023-08-11T15:44:54.194494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:16:51.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189482+0000","last_change":"2023-08-11T15:44:19.863724+0000","last_active":"2023-08-11T15:44:54.189482+0000","last_peered":"2023-08-11T15:44:54.189482+0000","last_clean":"2023-08-11T15:44:54.189482+0000","last_became_active":"2023-08-11T15:44:19.863268+0000","last_became_peered":"2023-08-11T15:44:19.863268+0000","last_unstale":"2023-08-11T15:44:54.189482+0000","last_undegraded":"2023-08-11T15:44:54.189482+0000","last_fullsized":"2023-08-11T15:44:54.189482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:27:34.481889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497528+0000","last_change":"2023-08-11T15:44:24.191301+0000","last_active":"2023-08-11T15:44:27.497528+0000","last_peered":"2023-08-11T15:44:27.497528+0000","last_clean":"2023-08-11T15:44:27.497528+0000","last_became_active":"2023-08-11T15:44:24.190686+0000","last_became_peered":"2023-08-11T15:44:24.190686+0000","last_unstale":"2023-08-11T15:44:27.497528+0000","last_undegraded":"2023-08-11T15:44:27.497528+0000","last_fullsized":"2023-08-11T15:44:27.497528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:29:52.896841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130772+0000","last_change":"2023-08-11T15:44:19.147756+0000","last_active":"2023-08-11T15:44:27.130772+0000","last_peered":"2023-08-11T15:44:27.130772+0000","last_clean":"2023-08-11T15:44:27.130772+0000","last_became_active":"2023-08-11T15:44:19.147426+0000","last_became_peered":"2023-08-11T15:44:19.147426+0000","last_unstale":"2023-08-11T15:44:27.130772+0000","last_undegraded":"2023-08-11T15:44:27.130772+0000","last_fullsized":"2023-08-11T15:44:27.130772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:45.863313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.190289+0000","last_change":"2023-08-11T15:44:19.863354+0000","last_active":"2023-08-11T15:44:54.190289+0000","last_peered":"2023-08-11T15:44:54.190289+0000","last_clean":"2023-08-11T15:44:54.190289+0000","last_became_active":"2023-08-11T15:44:19.862700+0000","last_became_peered":"2023-08-11T15:44:19.862700+0000","last_unstale":"2023-08-11T15:44:54.190289+0000","last_undegraded":"2023-08-11T15:44:54.190289+0000","last_fullsized":"2023-08-11T15:44:54.190289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:14:44.778310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.240828+0000","last_change":"2023-08-11T15:44:24.204657+0000","last_active":"2023-08-11T15:44:26.240828+0000","last_peered":"2023-08-11T15:44:26.240828+0000","last_clean":"2023-08-11T15:44:26.240828+0000","last_became_active":"2023-08-11T15:44:24.204108+0000","last_became_peered":"2023-08-11T15:44:24.204108+0000","last_unstale":"2023-08-11T15:44:26.240828+0000","last_undegraded":"2023-08-11T15:44:26.240828+0000","last_fullsized":"2023-08-11T15:44:26.240828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:17.431253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.044145+0000","last_change":"2023-08-11T15:44:19.147014+0000","last_active":"2023-08-11T15:44:27.044145+0000","last_peered":"2023-08-11T15:44:27.044145+0000","last_clean":"2023-08-11T15:44:27.044145+0000","last_became_active":"2023-08-11T15:44:19.146720+0000","last_became_peered":"2023-08-11T15:44:19.146720+0000","last_unstale":"2023-08-11T15:44:27.044145+0000","last_undegraded":"2023-08-11T15:44:27.044145+0000","last_fullsized":"2023-08-11T15:44:27.044145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:42:56.620766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T15:44:27.498380+0000","last_change":"2023-08-11T15:44:54.190492+0000","last_active":"2023-08-11T15:44:54.190492+0000","last_peered":"2023-08-11T15:44:27.498380+0000","last_clean":"2023-08-11T15:44:27.498380+0000","last_became_active":"2023-08-11T15:44:19.853078+0000","last_became_peered":"2023-08-11T15:44:19.853078+0000","last_unstale":"2023-08-11T15:44:27.498380+0000","last_undegraded":"2023-08-11T15:44:27.498380+0000","last_fullsized":"2023-08-11T15:44:27.498380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043195+0000","last_change":"2023-08-11T15:44:24.192709+0000","last_active":"2023-08-11T15:44:27.043195+0000","last_peered":"2023-08-11T15:44:27.043195+0000","last_clean":"2023-08-11T15:44:27.043195+0000","last_became_active":"2023-08-11T15:44:24.192297+0000","last_became_peered":"2023-08-11T15:44:24.192297+0000","last_unstale":"2023-08-11T15:44:27.043195+0000","last_undegraded":"2023-08-11T15:44:27.043195+0000","last_fullsized":"2023-08-11T15:44:27.043195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:39.723615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.399018+0000","last_change":"2023-08-11T15:44:19.158301+0000","last_active":"2023-08-11T15:44:25.399018+0000","last_peered":"2023-08-11T15:44:25.399018+0000","last_clean":"2023-08-11T15:44:25.399018+0000","last_became_active":"2023-08-11T15:44:19.157990+0000","last_became_peered":"2023-08-11T15:44:19.157990+0000","last_unstale":"2023-08-11T15:44:25.399018+0000","last_undegraded":"2023-08-11T15:44:25.399018+0000","last_fullsized":"2023-08-11T15:44:25.399018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:50:02.696501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T15:44:54.189944+0000","last_change":"2023-08-11T15:44:19.857357+0000","last_active":"2023-08-11T15:44:54.189944+0000","last_peered":"2023-08-11T15:44:54.189944+0000","last_clean":"2023-08-11T15:44:54.189944+0000","last_became_active":"2023-08-11T15:44:19.857020+0000","last_became_peered":"2023-08-11T15:44:19.857020+0000","last_unstale":"2023-08-11T15:44:54.189944+0000","last_undegraded":"2023-08-11T15:44:54.189944+0000","last_fullsized":"2023-08-11T15:44:54.189944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:37.243586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.183978+0000","last_change":"2023-08-11T15:44:19.863314+0000","last_active":"2023-08-11T15:44:25.183978+0000","last_peered":"2023-08-11T15:44:25.183978+0000","last_clean":"2023-08-11T15:44:25.183978+0000","last_became_active":"2023-08-11T15:44:19.862964+0000","last_became_peered":"2023-08-11T15:44:19.862964+0000","last_unstale":"2023-08-11T15:44:25.183978+0000","last_undegraded":"2023-08-11T15:44:25.183978+0000","last_fullsized":"2023-08-11T15:44:25.183978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:16.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130670+0000","last_change":"2023-08-11T15:44:19.850575+0000","last_active":"2023-08-11T15:44:27.130670+0000","last_peered":"2023-08-11T15:44:27.130670+0000","last_clean":"2023-08-11T15:44:27.130670+0000","last_became_active":"2023-08-11T15:44:19.850231+0000","last_became_peered":"2023-08-11T15:44:19.850231+0000","last_unstale":"2023-08-11T15:44:27.130670+0000","last_undegraded":"2023-08-11T15:44:27.130670+0000","last_fullsized":"2023-08-11T15:44:27.130670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:04:21.005350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130303+0000","last_change":"2023-08-11T15:44:19.857387+0000","last_active":"2023-08-11T15:44:27.130303+0000","last_peered":"2023-08-11T15:44:27.130303+0000","last_clean":"2023-08-11T15:44:27.130303+0000","last_became_active":"2023-08-11T15:44:19.856830+0000","last_became_peered":"2023-08-11T15:44:19.856830+0000","last_unstale":"2023-08-11T15:44:27.130303+0000","last_undegraded":"2023-08-11T15:44:27.130303+0000","last_fullsized":"2023-08-11T15:44:27.130303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:42.512448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:26.241040+0000","last_change":"2023-08-11T15:44:19.868112+0000","last_active":"2023-08-11T15:44:26.241040+0000","last_peered":"2023-08-11T15:44:26.241040+0000","last_clean":"2023-08-11T15:44:26.241040+0000","last_became_active":"2023-08-11T15:44:19.867610+0000","last_became_peered":"2023-08-11T15:44:19.867610+0000","last_unstale":"2023-08-11T15:44:26.241040+0000","last_undegraded":"2023-08-11T15:44:26.241040+0000","last_fullsized":"2023-08-11T15:44:26.241040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:18.810119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398997+0000","last_change":"2023-08-11T15:44:19.858349+0000","last_active":"2023-08-11T15:44:25.398997+0000","last_peered":"2023-08-11T15:44:25.398997+0000","last_clean":"2023-08-11T15:44:25.398997+0000","last_became_active":"2023-08-11T15:44:19.858041+0000","last_became_peered":"2023-08-11T15:44:19.858041+0000","last_unstale":"2023-08-11T15:44:25.398997+0000","last_undegraded":"2023-08-11T15:44:25.398997+0000","last_fullsized":"2023-08-11T15:44:25.398997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:24:04.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184464+0000","last_change":"2023-08-11T15:44:19.865383+0000","last_active":"2023-08-11T15:44:25.184464+0000","last_peered":"2023-08-11T15:44:25.184464+0000","last_clean":"2023-08-11T15:44:25.184464+0000","last_became_active":"2023-08-11T15:44:19.865065+0000","last_became_peered":"2023-08-11T15:44:19.865065+0000","last_unstale":"2023-08-11T15:44:25.184464+0000","last_undegraded":"2023-08-11T15:44:25.184464+0000","last_fullsized":"2023-08-11T15:44:25.184464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:39:27.425778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.130961+0000","last_change":"2023-08-11T15:44:19.859297+0000","last_active":"2023-08-11T15:44:27.130961+0000","last_peered":"2023-08-11T15:44:27.130961+0000","last_clean":"2023-08-11T15:44:27.130961+0000","last_became_active":"2023-08-11T15:44:19.857517+0000","last_became_peered":"2023-08-11T15:44:19.857517+0000","last_unstale":"2023-08-11T15:44:27.130961+0000","last_undegraded":"2023-08-11T15:44:27.130961+0000","last_fullsized":"2023-08-11T15:44:27.130961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:52:53.104396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.179277+0000","last_change":"2023-08-11T15:44:19.850485+0000","last_active":"2023-08-11T15:44:25.179277+0000","last_peered":"2023-08-11T15:44:25.179277+0000","last_clean":"2023-08-11T15:44:25.179277+0000","last_became_active":"2023-08-11T15:44:19.850047+0000","last_became_peered":"2023-08-11T15:44:19.850047+0000","last_unstale":"2023-08-11T15:44:25.179277+0000","last_undegraded":"2023-08-11T15:44:25.179277+0000","last_fullsized":"2023-08-11T15:44:25.179277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:18:11.380061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.043361+0000","last_change":"2023-08-11T15:44:19.861953+0000","last_active":"2023-08-11T15:44:27.043361+0000","last_peered":"2023-08-11T15:44:27.043361+0000","last_clean":"2023-08-11T15:44:27.043361+0000","last_became_active":"2023-08-11T15:44:19.861523+0000","last_became_peered":"2023-08-11T15:44:19.861523+0000","last_unstale":"2023-08-11T15:44:27.043361+0000","last_undegraded":"2023-08-11T15:44:27.043361+0000","last_fullsized":"2023-08-11T15:44:27.043361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:29:14.229621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.178755+0000","last_change":"2023-08-11T15:44:19.849283+0000","last_active":"2023-08-11T15:44:25.178755+0000","last_peered":"2023-08-11T15:44:25.178755+0000","last_clean":"2023-08-11T15:44:25.178755+0000","last_became_active":"2023-08-11T15:44:19.848304+0000","last_became_peered":"2023-08-11T15:44:19.848304+0000","last_unstale":"2023-08-11T15:44:25.178755+0000","last_undegraded":"2023-08-11T15:44:25.178755+0000","last_fullsized":"2023-08-11T15:44:25.178755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:06.449360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:27.497606+0000","last_change":"2023-08-11T15:44:19.853532+0000","last_active":"2023-08-11T15:44:27.497606+0000","last_peered":"2023-08-11T15:44:27.497606+0000","last_clean":"2023-08-11T15:44:27.497606+0000","last_became_active":"2023-08-11T15:44:19.853182+0000","last_became_peered":"2023-08-11T15:44:19.853182+0000","last_unstale":"2023-08-11T15:44:27.497606+0000","last_undegraded":"2023-08-11T15:44:27.497606+0000","last_fullsized":"2023-08-11T15:44:27.497606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:31:51.245443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.398519+0000","last_change":"2023-08-11T15:44:19.860571+0000","last_active":"2023-08-11T15:44:25.398519+0000","last_peered":"2023-08-11T15:44:25.398519+0000","last_clean":"2023-08-11T15:44:25.398519+0000","last_became_active":"2023-08-11T15:44:19.858570+0000","last_became_peered":"2023-08-11T15:44:19.858570+0000","last_unstale":"2023-08-11T15:44:25.398519+0000","last_undegraded":"2023-08-11T15:44:25.398519+0000","last_fullsized":"2023-08-11T15:44:25.398519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:08:37.343811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T15:44:25.184137+0000","last_change":"2023-08-11T15:44:19.865988+0000","last_active":"2023-08-11T15:44:25.184137+0000","last_peered":"2023-08-11T15:44:25.184137+0000","last_clean":"2023-08-11T15:44:25.184137+0000","last_became_active":"2023-08-11T15:44:19.865613+0000","last_became_peered":"2023-08-11T15:44:19.865613+0000","last_unstale":"2023-08-11T15:44:25.184137+0000","last_undegraded":"2023-08-11T15:44:25.184137+0000","last_fullsized":"2023-08-11T15:44:25.184137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:07.472727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":48682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.77400000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1061196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.627}]},{"osd":3,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.503}]},{"osd":4,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38400000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.497}]},{"osd":8,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.61299999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83399999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.627},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.751}]},{"osd":6,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.67100000000000004}]},{"osd":8,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.80300000000000005}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":481002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.75800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72499999999999998,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.85799999999999998,"5min":1.05,"15min":1.05},"last":1.1160000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.435,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99199999999999999,"5min":1.077,"15min":1.077},"last":1.145},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0980000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.96899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95399999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":1.1859999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.5819999999999999,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":1.044}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.1299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1075775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.45500000000000002,"15min":0.44800000000000001},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.628},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Aug 11 15:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.79300000000000004,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.498},"max":{"1min":4.8890000000000002,"5min":4.8890000000000002,"15min":4.8890000000000002},"last":0.88}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":2.077,"15min":2.077},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.008,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.581,"15min":1.581},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92500000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.7,"15min":1.7},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0269999999999999,"5min":1.22,"15min":1.22},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1319999999999999,"5min":1.623,"15min":1.623},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.61799999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.63,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.90000000000000002,"15min":0.89200000000000002},"min":{"1min":0.64500000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.85499999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.98399999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.95199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":470965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.76600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88600000000000001,"5min":4.9020000000000001,"15min":4.9020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.45900000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.62}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80800000000000005,"5min":1.593,"15min":1.593},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 15:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.024,"15min":1.024},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74099999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":1.05,"15min":1.05},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.90100000000000002}]},{"osd":8,"last update":"Fri Aug 11 15:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67700000000000005,"15min":0.66600000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1180000000000001,"5min":1.298,"15min":1.298},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.5409999999999999,"5min":6.0609999999999999,"15min":6.0609999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.087,"5min":6.101,"15min":6.101},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.621,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.441,"15min":0.433},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46100000000000002,"15min":0.44700000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.65300000000000002,"15min":0.61299999999999999},"min":{"1min":0.56000000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.54300000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.056,"5min":1.175,"15min":1.175},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.032,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.74399999999999999,"15min":0.72799999999999998},"min":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.78900000000000003,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.85799999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1779999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.149,"5min":1.276,"15min":1.276},"last":1.0489999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.51400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.81999999999999995}]},{"osd":11,"last update":"Fri Aug 11 15:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.016}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":470999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54600000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80300000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.44},"min":{"1min":0.29699999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69499999999999995}]},{"osd":8,"last update":"Fri Aug 11 15:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.879},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.97499999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.95999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":467227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.47699999999999998,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68100000000000005,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.746}]},{"osd":2,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55600000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94399999999999995,"5min":1.105,"15min":1.105},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.53800000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.72399999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.58799999999999997,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Aug 11 15:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.629},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.73599999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95099999999999996,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.266,"5min":5.266,"15min":5.266},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.51000000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.84799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":467145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.317},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.35999999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83599999999999997,"5min":1.552,"15min":1.552},"last":0.626},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":1.502,"15min":1.502},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47499999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.016,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93899999999999995,"5min":1.526,"15min":1.526},"last":0.68999999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1057342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79500000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60599999999999998,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83299999999999996,"5min":5.359,"15min":5.359},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.434,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.872,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.443},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77000000000000002,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.36899999999999999},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41999999999999998,"15min":0.40400000000000003},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.73099999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.032,"15min":1.032},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82599999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58599999999999997}]},{"osd":8,"last update":"Fri Aug 11 15:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.71599999999999997}]},{"osd":10,"last update":"Fri Aug 11 15:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":21257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:44:58.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:58 smithi134 ceph-mon[130447]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:58 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3895105932' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T15:44:58.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:58 smithi032 ceph-mon[97103]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:58.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:58 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3895105932' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T15:44:58.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:58 smithi192 ceph-mon[130806]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:58.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:58 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3895105932' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T15:44:59.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:59 smithi134 ceph-mon[130447]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:59.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:44:59 smithi134 ceph-mon[130447]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:44:59.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:59 smithi032 ceph-mon[97103]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:59.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:44:59 smithi032 ceph-mon[97103]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:44:59.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:59 smithi192 ceph-mon[130806]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:44:59.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:44:59 smithi192 ceph-mon[130806]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:00.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T15:45:00.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.987+0000 7fc0e55ff700 1 -- 172.21.15.32:0/2143842415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0100920 msgr2=0x7fc0e0100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:00.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.987+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/2143842415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0100920 0x7fc0e0100d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00099c0 tx=0x7fc0d0009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:00.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.987+0000 7fc0e55ff700 1 -- 172.21.15.32:0/2143842415 shutdown_connections 2023-08-11T15:45:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.988+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/2143842415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0100920 0x7fc0e0100d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.988+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/2143842415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e01024b0 0x7fc0e0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.988+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/2143842415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e0101b10 0x7fc0e0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:00.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.988+0000 7fc0e55ff700 1 -- 172.21.15.32:0/2143842415 >> 172.21.15.32:0/2143842415 conn(0x7fc0e00fbe50 msgr2=0x7fc0e00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:00.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.988+0000 7fc0e55ff700 1 -- 172.21.15.32:0/2143842415 shutdown_connections 2023-08-11T15:45:00.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.988+0000 7fc0e55ff700 1 -- 172.21.15.32:0/2143842415 wait complete. 2023-08-11T15:45:00.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 Processor -- start 2023-08-11T15:45:00.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 -- start start 2023-08-11T15:45:00.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0100920 0x7fc0e0196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:00.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0101b10 0x7fc0e0196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:00.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e01024b0 0x7fc0e019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:01.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e019e190 con 0x7fc0e0101b10 2023-08-11T15:45:01.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e019e2d0 con 0x7fc0e01024b0 2023-08-11T15:45:01.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0e55ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e019e460 con 0x7fc0e0100920 2023-08-11T15:45:01.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0de7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0101b10 0x7fc0e0196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:01.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0100920 0x7fc0e0196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:01.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0100920 0x7fc0e0196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:40646/0 (socket says 172.21.15.32:40646) 2023-08-11T15:45:01.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.989+0000 7fc0deffd700 1 -- 172.21.15.32:0/880027191 learned_addr learned my addr 172.21.15.32:0/880027191 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:01.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0df7fe700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e01024b0 0x7fc0e019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:01.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0deffd700 1 -- 172.21.15.32:0/880027191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e01024b0 msgr2=0x7fc0e019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:01.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0deffd700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e01024b0 0x7fc0e019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:01.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0deffd700 1 -- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0101b10 msgr2=0x7fc0e0196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:01.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0deffd700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0101b10 0x7fc0e0196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:01.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0deffd700 1 -- 172.21.15.32:0/880027191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00095f0 con 0x7fc0e0100920 2023-08-11T15:45:01.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0df7fe700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e01024b0 0x7fc0e019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:45:01.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0deffd700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0100920 0x7fc0e0196610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009990 tx=0x7fc0d000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:01.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.990+0000 7fc0c7fff700 1 -- 172.21.15.32:0/880027191 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d001d070 con 0x7fc0e0100920 2023-08-11T15:45:01.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.991+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e019e6e0 con 0x7fc0e0100920 2023-08-11T15:45:01.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.991+0000 7fc0c7fff700 1 -- 172.21.15.32:0/880027191 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d00054f0 con 0x7fc0e0100920 2023-08-11T15:45:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.991+0000 7fc0c7fff700 1 -- 172.21.15.32:0/880027191 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0004b00 con 0x7fc0e0100920 2023-08-11T15:45:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.991+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e019eb60 con 0x7fc0e0100920 2023-08-11T15:45:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.992+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e010c180 con 0x7fc0e0100920 2023-08-11T15:45:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.994+0000 7fc0c7fff700 1 -- 172.21.15.32:0/880027191 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc0d000fb80 con 0x7fc0e0100920 2023-08-11T15:45:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.995+0000 7fc0c7fff700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc0c80a0020 0x7fc0c80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.995+0000 7fc0c7fff700 1 -- 172.21.15.32:0/880027191 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc0d00cc510 con 0x7fc0e0100920 2023-08-11T15:45:01.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.995+0000 7fc0de7fc700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc0c80a0020 0x7fc0c80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:01.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.996+0000 7fc0de7fc700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc0c80a0020 0x7fc0c80a24e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009c60 tx=0x7fc0d4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:01.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:00.997+0000 7fc0c7fff700 1 -- 172.21.15.32:0/880027191 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc0d0095ef0 con 0x7fc0e0100920 2023-08-11T15:45:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.210+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0e0060d60 con 0x7fc0c80a0020 2023-08-11T15:45:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.223+0000 7fc0c7fff700 1 -- 172.21.15.32:0/880027191 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422383 (secure 0 0 0) 0x7fc0e0060d60 con 0x7fc0c80a0020 2023-08-11T15:45:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc0c80a0020 msgr2=0x7fc0c80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc0c80a0020 0x7fc0c80a24e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009c60 tx=0x7fc0d4009260 comp rx=0 tx=0).stop 2023-08-11T15:45:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0100920 msgr2=0x7fc0e0196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0100920 0x7fc0e0196610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009990 tx=0x7fc0d000f690 comp rx=0 tx=0).stop 2023-08-11T15:45:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 shutdown_connections 2023-08-11T15:45:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc0c80a0020 0x7fc0c80a24e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e01024b0 0x7fc0e019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0100920 0x7fc0e0196610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 --2- 172.21.15.32:0/880027191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e0101b10 0x7fc0e0196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 >> 172.21.15.32:0/880027191 conn(0x7fc0e00fbe50 msgr2=0x7fc0e00fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 shutdown_connections 2023-08-11T15:45:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:01.229+0000 7fc0e55ff700 1 -- 172.21.15.32:0/880027191 wait complete. 2023-08-11T15:45:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-08-11T15:45:01.251 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-08-11T15:45:00.076925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":635173,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":161,"num_read_kb":177,"num_write":244,"num_write_kb":3003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":423,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337912,"kb_used_data":13640,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603912,"statfs":{"total":1151940427776,"available":1151594405888,"internally_reserved":0,"allocated":13967360,"data_stored":7771040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40165,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":57,"num_write":14,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002196"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192439+0000","last_change":"2023-08-11T15:44:19.862927+0000","last_active":"2023-08-11T15:44:55.192439+0000","last_peered":"2023-08-11T15:44:55.192439+0000","last_clean":"2023-08-11T15:44:55.192439+0000","last_became_active":"2023-08-11T15:44:19.862626+0000","last_became_peered":"2023-08-11T15:44:19.862626+0000","last_unstale":"2023-08-11T15:44:55.192439+0000","last_undegraded":"2023-08-11T15:44:55.192439+0000","last_fullsized":"2023-08-11T15:44:55.192439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:39:26.212373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190047+0000","last_change":"2023-08-11T15:44:19.852455+0000","last_active":"2023-08-11T15:44:55.190047+0000","last_peered":"2023-08-11T15:44:55.190047+0000","last_clean":"2023-08-11T15:44:55.190047+0000","last_became_active":"2023-08-11T15:44:19.851897+0000","last_became_peered":"2023-08-11T15:44:19.851897+0000","last_unstale":"2023-08-11T15:44:55.190047+0000","last_undegraded":"2023-08-11T15:44:55.190047+0000","last_fullsized":"2023-08-11T15:44:55.190047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:23:56.473990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189740+0000","last_change":"2023-08-11T15:44:19.855232+0000","last_active":"2023-08-11T15:44:55.189740+0000","last_peered":"2023-08-11T15:44:55.189740+0000","last_clean":"2023-08-11T15:44:55.189740+0000","last_became_active":"2023-08-11T15:44:19.854940+0000","last_became_peered":"2023-08-11T15:44:19.854940+0000","last_unstale":"2023-08-11T15:44:55.189740+0000","last_undegraded":"2023-08-11T15:44:55.189740+0000","last_fullsized":"2023-08-11T15:44:55.189740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:58:15.175180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198360+0000","last_change":"2023-08-11T15:44:19.856137+0000","last_active":"2023-08-11T15:44:55.198360+0000","last_peered":"2023-08-11T15:44:55.198360+0000","last_clean":"2023-08-11T15:44:55.198360+0000","last_became_active":"2023-08-11T15:44:19.855852+0000","last_became_peered":"2023-08-11T15:44:19.855852+0000","last_unstale":"2023-08-11T15:44:55.198360+0000","last_undegraded":"2023-08-11T15:44:55.198360+0000","last_fullsized":"2023-08-11T15:44:55.198360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:50:01.697765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194593+0000","last_change":"2023-08-11T15:44:19.862618+0000","last_active":"2023-08-11T15:44:55.194593+0000","last_peered":"2023-08-11T15:44:55.194593+0000","last_clean":"2023-08-11T15:44:55.194593+0000","last_became_active":"2023-08-11T15:44:19.862265+0000","last_became_peered":"2023-08-11T15:44:19.862265+0000","last_unstale":"2023-08-11T15:44:55.194593+0000","last_undegraded":"2023-08-11T15:44:55.194593+0000","last_fullsized":"2023-08-11T15:44:55.194593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:40.449974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192805+0000","last_change":"2023-08-11T15:44:19.857801+0000","last_active":"2023-08-11T15:44:55.192805+0000","last_peered":"2023-08-11T15:44:55.192805+0000","last_clean":"2023-08-11T15:44:55.192805+0000","last_became_active":"2023-08-11T15:44:19.857388+0000","last_became_peered":"2023-08-11T15:44:19.857388+0000","last_unstale":"2023-08-11T15:44:55.192805+0000","last_undegraded":"2023-08-11T15:44:55.192805+0000","last_fullsized":"2023-08-11T15:44:55.192805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:09:13.938322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194812+0000","last_change":"2023-08-11T15:44:19.859334+0000","last_active":"2023-08-11T15:44:55.194812+0000","last_peered":"2023-08-11T15:44:55.194812+0000","last_clean":"2023-08-11T15:44:55.194812+0000","last_became_active":"2023-08-11T15:44:19.859073+0000","last_became_peered":"2023-08-11T15:44:19.859073+0000","last_unstale":"2023-08-11T15:44:55.194812+0000","last_undegraded":"2023-08-11T15:44:55.194812+0000","last_fullsized":"2023-08-11T15:44:55.194812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:40:27.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194562+0000","last_change":"2023-08-11T15:44:19.853064+0000","last_active":"2023-08-11T15:44:55.194562+0000","last_peered":"2023-08-11T15:44:55.194562+0000","last_clean":"2023-08-11T15:44:55.194562+0000","last_became_active":"2023-08-11T15:44:19.852736+0000","last_became_peered":"2023-08-11T15:44:19.852736+0000","last_unstale":"2023-08-11T15:44:55.194562+0000","last_undegraded":"2023-08-11T15:44:55.194562+0000","last_fullsized":"2023-08-11T15:44:55.194562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:57:13.446578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191401+0000","last_change":"2023-08-11T15:44:19.861912+0000","last_active":"2023-08-11T15:44:55.191401+0000","last_peered":"2023-08-11T15:44:55.191401+0000","last_clean":"2023-08-11T15:44:55.191401+0000","last_became_active":"2023-08-11T15:44:19.861374+0000","last_became_peered":"2023-08-11T15:44:19.861374+0000","last_unstale":"2023-08-11T15:44:55.191401+0000","last_undegraded":"2023-08-11T15:44:55.191401+0000","last_fullsized":"2023-08-11T15:44:55.191401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:43:22.890968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193523+0000","last_change":"2023-08-11T15:44:19.858862+0000","last_active":"2023-08-11T15:44:55.193523+0000","last_peered":"2023-08-11T15:44:55.193523+0000","last_clean":"2023-08-11T15:44:55.193523+0000","last_became_active":"2023-08-11T15:44:19.858512+0000","last_became_peered":"2023-08-11T15:44:19.858512+0000","last_unstale":"2023-08-11T15:44:55.193523+0000","last_undegraded":"2023-08-11T15:44:55.193523+0000","last_fullsized":"2023-08-11T15:44:55.193523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:40:43.630548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195034+0000","last_change":"2023-08-11T15:44:19.864063+0000","last_active":"2023-08-11T15:44:55.195034+0000","last_peered":"2023-08-11T15:44:55.195034+0000","last_clean":"2023-08-11T15:44:55.195034+0000","last_became_active":"2023-08-11T15:44:19.863081+0000","last_became_peered":"2023-08-11T15:44:19.863081+0000","last_unstale":"2023-08-11T15:44:55.195034+0000","last_undegraded":"2023-08-11T15:44:55.195034+0000","last_fullsized":"2023-08-11T15:44:55.195034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:58:18.278385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198186+0000","last_change":"2023-08-11T15:44:19.855347+0000","last_active":"2023-08-11T15:44:55.198186+0000","last_peered":"2023-08-11T15:44:55.198186+0000","last_clean":"2023-08-11T15:44:55.198186+0000","last_became_active":"2023-08-11T15:44:19.855025+0000","last_became_peered":"2023-08-11T15:44:19.855025+0000","last_unstale":"2023-08-11T15:44:55.198186+0000","last_undegraded":"2023-08-11T15:44:55.198186+0000","last_fullsized":"2023-08-11T15:44:55.198186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:44:49.739947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195827+0000","last_change":"2023-08-11T15:44:19.862153+0000","last_active":"2023-08-11T15:44:55.195827+0000","last_peered":"2023-08-11T15:44:55.195827+0000","last_clean":"2023-08-11T15:44:55.195827+0000","last_became_active":"2023-08-11T15:44:19.861830+0000","last_became_peered":"2023-08-11T15:44:19.861830+0000","last_unstale":"2023-08-11T15:44:55.195827+0000","last_undegraded":"2023-08-11T15:44:55.195827+0000","last_fullsized":"2023-08-11T15:44:55.195827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:16:01.026944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189564+0000","last_change":"2023-08-11T15:44:19.857184+0000","last_active":"2023-08-11T15:44:55.189564+0000","last_peered":"2023-08-11T15:44:55.189564+0000","last_clean":"2023-08-11T15:44:55.189564+0000","last_became_active":"2023-08-11T15:44:19.856898+0000","last_became_peered":"2023-08-11T15:44:19.856898+0000","last_unstale":"2023-08-11T15:44:55.189564+0000","last_undegraded":"2023-08-11T15:44:55.189564+0000","last_fullsized":"2023-08-11T15:44:55.189564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:35:04.984315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189495+0000","last_change":"2023-08-11T15:44:19.852808+0000","last_active":"2023-08-11T15:44:55.189495+0000","last_peered":"2023-08-11T15:44:55.189495+0000","last_clean":"2023-08-11T15:44:55.189495+0000","last_became_active":"2023-08-11T15:44:19.852477+0000","last_became_peered":"2023-08-11T15:44:19.852477+0000","last_unstale":"2023-08-11T15:44:55.189495+0000","last_undegraded":"2023-08-11T15:44:55.189495+0000","last_fullsized":"2023-08-11T15:44:55.189495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:15.737333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190450+0000","last_change":"2023-08-11T15:44:19.856273+0000","last_active":"2023-08-11T15:44:55.190450+0000","last_peered":"2023-08-11T15:44:55.190450+0000","last_clean":"2023-08-11T15:44:55.190450+0000","last_became_active":"2023-08-11T15:44:19.854579+0000","last_became_peered":"2023-08-11T15:44:19.854579+0000","last_unstale":"2023-08-11T15:44:55.190450+0000","last_undegraded":"2023-08-11T15:44:55.190450+0000","last_fullsized":"2023-08-11T15:44:55.190450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:02:07.058213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192239+0000","last_change":"2023-08-11T15:44:19.866820+0000","last_active":"2023-08-11T15:44:55.192239+0000","last_peered":"2023-08-11T15:44:55.192239+0000","last_clean":"2023-08-11T15:44:55.192239+0000","last_became_active":"2023-08-11T15:44:19.866507+0000","last_became_peered":"2023-08-11T15:44:19.866507+0000","last_unstale":"2023-08-11T15:44:55.192239+0000","last_undegraded":"2023-08-11T15:44:55.192239+0000","last_fullsized":"2023-08-11T15:44:55.192239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:41:54.967368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193782+0000","last_change":"2023-08-11T15:44:19.865821+0000","last_active":"2023-08-11T15:44:55.193782+0000","last_peered":"2023-08-11T15:44:55.193782+0000","last_clean":"2023-08-11T15:44:55.193782+0000","last_became_active":"2023-08-11T15:44:19.865510+0000","last_became_peered":"2023-08-11T15:44:19.865510+0000","last_unstale":"2023-08-11T15:44:55.193782+0000","last_undegraded":"2023-08-11T15:44:55.193782+0000","last_fullsized":"2023-08-11T15:44:55.193782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:20:32.474091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194714+0000","last_change":"2023-08-11T15:44:19.863079+0000","last_active":"2023-08-11T15:44:55.194714+0000","last_peered":"2023-08-11T15:44:55.194714+0000","last_clean":"2023-08-11T15:44:55.194714+0000","last_became_active":"2023-08-11T15:44:19.862663+0000","last_became_peered":"2023-08-11T15:44:19.862663+0000","last_unstale":"2023-08-11T15:44:55.194714+0000","last_undegraded":"2023-08-11T15:44:55.194714+0000","last_fullsized":"2023-08-11T15:44:55.194714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:41:51.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194903+0000","last_change":"2023-08-11T15:44:24.201308+0000","last_active":"2023-08-11T15:44:55.194903+0000","last_peered":"2023-08-11T15:44:55.194903+0000","last_clean":"2023-08-11T15:44:55.194903+0000","last_became_active":"2023-08-11T15:44:24.200884+0000","last_became_peered":"2023-08-11T15:44:24.200884+0000","last_unstale":"2023-08-11T15:44:55.194903+0000","last_undegraded":"2023-08-11T15:44:55.194903+0000","last_fullsized":"2023-08-11T15:44:55.194903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:59.419400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193546+0000","last_change":"2023-08-11T15:44:19.155621+0000","last_active":"2023-08-11T15:44:55.193546+0000","last_peered":"2023-08-11T15:44:55.193546+0000","last_clean":"2023-08-11T15:44:55.193546+0000","last_became_active":"2023-08-11T15:44:19.154494+0000","last_became_peered":"2023-08-11T15:44:19.154494+0000","last_unstale":"2023-08-11T15:44:55.193546+0000","last_undegraded":"2023-08-11T15:44:55.193546+0000","last_fullsized":"2023-08-11T15:44:55.193546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:09.243769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196857+0000","last_change":"2023-08-11T15:44:19.853101+0000","last_active":"2023-08-11T15:44:55.196857+0000","last_peered":"2023-08-11T15:44:55.196857+0000","last_clean":"2023-08-11T15:44:55.196857+0000","last_became_active":"2023-08-11T15:44:19.852804+0000","last_became_peered":"2023-08-11T15:44:19.852804+0000","last_unstale":"2023-08-11T15:44:55.196857+0000","last_undegraded":"2023-08-11T15:44:55.196857+0000","last_fullsized":"2023-08-11T15:44:55.196857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:26:49.173800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200726+0000","last_change":"2023-08-11T15:44:55.200726+0000","last_active":"2023-08-11T15:44:55.200726+0000","last_peered":"2023-08-11T15:44:55.200726+0000","last_clean":"2023-08-11T15:44:55.200726+0000","last_became_active":"2023-08-11T15:44:55.200360+0000","last_became_peered":"2023-08-11T15:44:55.200360+0000","last_unstale":"2023-08-11T15:44:55.200726+0000","last_undegraded":"2023-08-11T15:44:55.200726+0000","last_fullsized":"2023-08-11T15:44:55.200726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:59:09.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196984+0000","last_change":"2023-08-11T15:44:19.147219+0000","last_active":"2023-08-11T15:44:55.196984+0000","last_peered":"2023-08-11T15:44:55.196984+0000","last_clean":"2023-08-11T15:44:55.196984+0000","last_became_active":"2023-08-11T15:44:19.146874+0000","last_became_peered":"2023-08-11T15:44:19.146874+0000","last_unstale":"2023-08-11T15:44:55.196984+0000","last_undegraded":"2023-08-11T15:44:55.196984+0000","last_fullsized":"2023-08-11T15:44:55.196984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:33.758701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189983+0000","last_change":"2023-08-11T15:44:19.854103+0000","last_active":"2023-08-11T15:44:55.189983+0000","last_peered":"2023-08-11T15:44:55.189983+0000","last_clean":"2023-08-11T15:44:55.189983+0000","last_became_active":"2023-08-11T15:44:19.853542+0000","last_became_peered":"2023-08-11T15:44:19.853542+0000","last_unstale":"2023-08-11T15:44:55.189983+0000","last_undegraded":"2023-08-11T15:44:55.189983+0000","last_fullsized":"2023-08-11T15:44:55.189983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:35:27.247168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200630+0000","last_change":"2023-08-11T15:44:55.200630+0000","last_active":"2023-08-11T15:44:55.200630+0000","last_peered":"2023-08-11T15:44:55.200630+0000","last_clean":"2023-08-11T15:44:55.200630+0000","last_became_active":"2023-08-11T15:44:55.200271+0000","last_became_peered":"2023-08-11T15:44:55.200271+0000","last_unstale":"2023-08-11T15:44:55.200630+0000","last_undegraded":"2023-08-11T15:44:55.200630+0000","last_fullsized":"2023-08-11T15:44:55.200630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:50:50.162322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195272+0000","last_change":"2023-08-11T15:44:19.148019+0000","last_active":"2023-08-11T15:44:55.195272+0000","last_peered":"2023-08-11T15:44:55.195272+0000","last_clean":"2023-08-11T15:44:55.195272+0000","last_became_active":"2023-08-11T15:44:19.147501+0000","last_became_peered":"2023-08-11T15:44:19.147501+0000","last_unstale":"2023-08-11T15:44:55.195272+0000","last_undegraded":"2023-08-11T15:44:55.195272+0000","last_fullsized":"2023-08-11T15:44:55.195272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:32.333344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190689+0000","last_change":"2023-08-11T15:44:19.865230+0000","last_active":"2023-08-11T15:44:55.190689+0000","last_peered":"2023-08-11T15:44:55.190689+0000","last_clean":"2023-08-11T15:44:55.190689+0000","last_became_active":"2023-08-11T15:44:19.864838+0000","last_became_peered":"2023-08-11T15:44:19.864838+0000","last_unstale":"2023-08-11T15:44:55.190689+0000","last_undegraded":"2023-08-11T15:44:55.190689+0000","last_fullsized":"2023-08-11T15:44:55.190689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:10.222090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198211+0000","last_change":"2023-08-11T15:44:24.192282+0000","last_active":"2023-08-11T15:44:55.198211+0000","last_peered":"2023-08-11T15:44:55.198211+0000","last_clean":"2023-08-11T15:44:55.198211+0000","last_became_active":"2023-08-11T15:44:24.191972+0000","last_became_peered":"2023-08-11T15:44:24.191972+0000","last_unstale":"2023-08-11T15:44:55.198211+0000","last_undegraded":"2023-08-11T15:44:55.198211+0000","last_fullsized":"2023-08-11T15:44:55.198211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:14:12.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195406+0000","last_change":"2023-08-11T15:44:19.147181+0000","last_active":"2023-08-11T15:44:55.195406+0000","last_peered":"2023-08-11T15:44:55.195406+0000","last_clean":"2023-08-11T15:44:55.195406+0000","last_became_active":"2023-08-11T15:44:19.146843+0000","last_became_peered":"2023-08-11T15:44:19.146843+0000","last_unstale":"2023-08-11T15:44:55.195406+0000","last_undegraded":"2023-08-11T15:44:55.195406+0000","last_fullsized":"2023-08-11T15:44:55.195406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:39:50.508765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200249+0000","last_change":"2023-08-11T15:44:55.199783+0000","last_active":"2023-08-11T15:44:55.200249+0000","last_peered":"2023-08-11T15:44:55.200249+0000","last_clean":"2023-08-11T15:44:55.200249+0000","last_became_active":"2023-08-11T15:44:55.199391+0000","last_became_peered":"2023-08-11T15:44:55.199391+0000","last_unstale":"2023-08-11T15:44:55.200249+0000","last_undegraded":"2023-08-11T15:44:55.200249+0000","last_fullsized":"2023-08-11T15:44:55.200249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:19:02.132279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194561+0000","last_change":"2023-08-11T15:44:24.192621+0000","last_active":"2023-08-11T15:44:55.194561+0000","last_peered":"2023-08-11T15:44:55.194561+0000","last_clean":"2023-08-11T15:44:55.194561+0000","last_became_active":"2023-08-11T15:44:24.192183+0000","last_became_peered":"2023-08-11T15:44:24.192183+0000","last_unstale":"2023-08-11T15:44:55.194561+0000","last_undegraded":"2023-08-11T15:44:55.194561+0000","last_fullsized":"2023-08-11T15:44:55.194561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:16.822794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190900+0000","last_change":"2023-08-11T15:44:19.149437+0000","last_active":"2023-08-11T15:44:55.190900+0000","last_peered":"2023-08-11T15:44:55.190900+0000","last_clean":"2023-08-11T15:44:55.190900+0000","last_became_active":"2023-08-11T15:44:19.149086+0000","last_became_peered":"2023-08-11T15:44:19.149086+0000","last_unstale":"2023-08-11T15:44:55.190900+0000","last_undegraded":"2023-08-11T15:44:55.190900+0000","last_fullsized":"2023-08-11T15:44:55.190900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:50.716278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195215+0000","last_change":"2023-08-11T15:44:19.865452+0000","last_active":"2023-08-11T15:44:55.195215+0000","last_peered":"2023-08-11T15:44:55.195215+0000","last_clean":"2023-08-11T15:44:55.195215+0000","last_became_active":"2023-08-11T15:44:19.865133+0000","last_became_peered":"2023-08-11T15:44:19.865133+0000","last_unstale":"2023-08-11T15:44:55.195215+0000","last_undegraded":"2023-08-11T15:44:55.195215+0000","last_fullsized":"2023-08-11T15:44:55.195215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:42:54.760249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195540+0000","last_change":"2023-08-11T15:44:24.201817+0000","last_active":"2023-08-11T15:44:55.195540+0000","last_peered":"2023-08-11T15:44:55.195540+0000","last_clean":"2023-08-11T15:44:55.195540+0000","last_became_active":"2023-08-11T15:44:24.200766+0000","last_became_peered":"2023-08-11T15:44:24.200766+0000","last_unstale":"2023-08-11T15:44:55.195540+0000","last_undegraded":"2023-08-11T15:44:55.195540+0000","last_fullsized":"2023-08-11T15:44:55.195540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:46:08.346089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192629+0000","last_change":"2023-08-11T15:44:19.151213+0000","last_active":"2023-08-11T15:44:55.192629+0000","last_peered":"2023-08-11T15:44:55.192629+0000","last_clean":"2023-08-11T15:44:55.192629+0000","last_became_active":"2023-08-11T15:44:19.150914+0000","last_became_peered":"2023-08-11T15:44:19.150914+0000","last_unstale":"2023-08-11T15:44:55.192629+0000","last_undegraded":"2023-08-11T15:44:55.192629+0000","last_fullsized":"2023-08-11T15:44:55.192629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:30.877228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191123+0000","last_change":"2023-08-11T15:44:19.858238+0000","last_active":"2023-08-11T15:44:55.191123+0000","last_peered":"2023-08-11T15:44:55.191123+0000","last_clean":"2023-08-11T15:44:55.191123+0000","last_became_active":"2023-08-11T15:44:19.857919+0000","last_became_peered":"2023-08-11T15:44:19.857919+0000","last_unstale":"2023-08-11T15:44:55.191123+0000","last_undegraded":"2023-08-11T15:44:55.191123+0000","last_fullsized":"2023-08-11T15:44:55.191123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:15:28.098691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192085+0000","last_change":"2023-08-11T15:44:24.201485+0000","last_active":"2023-08-11T15:44:55.192085+0000","last_peered":"2023-08-11T15:44:55.192085+0000","last_clean":"2023-08-11T15:44:55.192085+0000","last_became_active":"2023-08-11T15:44:24.200984+0000","last_became_peered":"2023-08-11T15:44:24.200984+0000","last_unstale":"2023-08-11T15:44:55.192085+0000","last_undegraded":"2023-08-11T15:44:55.192085+0000","last_fullsized":"2023-08-11T15:44:55.192085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:57.566307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195775+0000","last_change":"2023-08-11T15:44:19.149289+0000","last_active":"2023-08-11T15:44:55.195775+0000","last_peered":"2023-08-11T15:44:55.195775+0000","last_clean":"2023-08-11T15:44:55.195775+0000","last_became_active":"2023-08-11T15:44:19.148881+0000","last_became_peered":"2023-08-11T15:44:19.148881+0000","last_unstale":"2023-08-11T15:44:55.195775+0000","last_undegraded":"2023-08-11T15:44:55.195775+0000","last_fullsized":"2023-08-11T15:44:55.195775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:02.163702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194495+0000","last_change":"2023-08-11T15:44:19.867860+0000","last_active":"2023-08-11T15:44:55.194495+0000","last_peered":"2023-08-11T15:44:55.194495+0000","last_clean":"2023-08-11T15:44:55.194495+0000","last_became_active":"2023-08-11T15:44:19.867545+0000","last_became_peered":"2023-08-11T15:44:19.867545+0000","last_unstale":"2023-08-11T15:44:55.194495+0000","last_undegraded":"2023-08-11T15:44:55.194495+0000","last_fullsized":"2023-08-11T15:44:55.194495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:46:29.660791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189976+0000","last_change":"2023-08-11T15:44:24.197251+0000","last_active":"2023-08-11T15:44:55.189976+0000","last_peered":"2023-08-11T15:44:55.189976+0000","last_clean":"2023-08-11T15:44:55.189976+0000","last_became_active":"2023-08-11T15:44:24.196735+0000","last_became_peered":"2023-08-11T15:44:24.196735+0000","last_unstale":"2023-08-11T15:44:55.189976+0000","last_undegraded":"2023-08-11T15:44:55.189976+0000","last_fullsized":"2023-08-11T15:44:55.189976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:04:00.057964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191849+0000","last_change":"2023-08-11T15:44:19.146196+0000","last_active":"2023-08-11T15:44:55.191849+0000","last_peered":"2023-08-11T15:44:55.191849+0000","last_clean":"2023-08-11T15:44:55.191849+0000","last_became_active":"2023-08-11T15:44:19.145812+0000","last_became_peered":"2023-08-11T15:44:19.145812+0000","last_unstale":"2023-08-11T15:44:55.191849+0000","last_undegraded":"2023-08-11T15:44:55.191849+0000","last_fullsized":"2023-08-11T15:44:55.191849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:51:59.428704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192831+0000","last_change":"2023-08-11T15:44:19.855389+0000","last_active":"2023-08-11T15:44:55.192831+0000","last_peered":"2023-08-11T15:44:55.192831+0000","last_clean":"2023-08-11T15:44:55.192831+0000","last_became_active":"2023-08-11T15:44:19.855063+0000","last_became_peered":"2023-08-11T15:44:19.855063+0000","last_unstale":"2023-08-11T15:44:55.192831+0000","last_undegraded":"2023-08-11T15:44:55.192831+0000","last_fullsized":"2023-08-11T15:44:55.192831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:42:37.301427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194782+0000","last_change":"2023-08-11T15:44:24.192450+0000","last_active":"2023-08-11T15:44:55.194782+0000","last_peered":"2023-08-11T15:44:55.194782+0000","last_clean":"2023-08-11T15:44:55.194782+0000","last_became_active":"2023-08-11T15:44:24.191989+0000","last_became_peered":"2023-08-11T15:44:24.191989+0000","last_unstale":"2023-08-11T15:44:55.194782+0000","last_undegraded":"2023-08-11T15:44:55.194782+0000","last_fullsized":"2023-08-11T15:44:55.194782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:06.182448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192857+0000","last_change":"2023-08-11T15:44:19.149002+0000","last_active":"2023-08-11T15:44:55.192857+0000","last_peered":"2023-08-11T15:44:55.192857+0000","last_clean":"2023-08-11T15:44:55.192857+0000","last_became_active":"2023-08-11T15:44:19.148635+0000","last_became_peered":"2023-08-11T15:44:19.148635+0000","last_unstale":"2023-08-11T15:44:55.192857+0000","last_undegraded":"2023-08-11T15:44:55.192857+0000","last_fullsized":"2023-08-11T15:44:55.192857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:58.450017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191583+0000","last_change":"2023-08-11T15:44:19.858067+0000","last_active":"2023-08-11T15:44:55.191583+0000","last_peered":"2023-08-11T15:44:55.191583+0000","last_clean":"2023-08-11T15:44:55.191583+0000","last_became_active":"2023-08-11T15:44:19.856641+0000","last_became_peered":"2023-08-11T15:44:19.856641+0000","last_unstale":"2023-08-11T15:44:55.191583+0000","last_undegraded":"2023-08-11T15:44:55.191583+0000","last_fullsized":"2023-08-11T15:44:55.191583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:17.780319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192698+0000","last_change":"2023-08-11T15:44:24.196806+0000","last_active":"2023-08-11T15:44:55.192698+0000","last_peered":"2023-08-11T15:44:55.192698+0000","last_clean":"2023-08-11T15:44:55.192698+0000","last_became_active":"2023-08-11T15:44:24.195268+0000","last_became_peered":"2023-08-11T15:44:24.195268+0000","last_unstale":"2023-08-11T15:44:55.192698+0000","last_undegraded":"2023-08-11T15:44:55.192698+0000","last_fullsized":"2023-08-11T15:44:55.192698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:47:43.203448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.202786+0000","last_change":"2023-08-11T15:44:55.202786+0000","last_active":"2023-08-11T15:44:55.202786+0000","last_peered":"2023-08-11T15:44:55.202786+0000","last_clean":"2023-08-11T15:44:55.202786+0000","last_became_active":"2023-08-11T15:44:55.202409+0000","last_became_peered":"2023-08-11T15:44:55.202409+0000","last_unstale":"2023-08-11T15:44:55.202786+0000","last_undegraded":"2023-08-11T15:44:55.202786+0000","last_fullsized":"2023-08-11T15:44:55.202786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:22.000673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190025+0000","last_change":"2023-08-11T15:44:19.853881+0000","last_active":"2023-08-11T15:44:55.190025+0000","last_peered":"2023-08-11T15:44:55.190025+0000","last_clean":"2023-08-11T15:44:55.190025+0000","last_became_active":"2023-08-11T15:44:19.853139+0000","last_became_peered":"2023-08-11T15:44:19.853139+0000","last_unstale":"2023-08-11T15:44:55.190025+0000","last_undegraded":"2023-08-11T15:44:55.190025+0000","last_fullsized":"2023-08-11T15:44:55.190025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:23:44.862419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197483+0000","last_change":"2023-08-11T15:44:24.201620+0000","last_active":"2023-08-11T15:44:55.197483+0000","last_peered":"2023-08-11T15:44:55.197483+0000","last_clean":"2023-08-11T15:44:55.197483+0000","last_became_active":"2023-08-11T15:44:24.201162+0000","last_became_peered":"2023-08-11T15:44:24.201162+0000","last_unstale":"2023-08-11T15:44:55.197483+0000","last_undegraded":"2023-08-11T15:44:55.197483+0000","last_fullsized":"2023-08-11T15:44:55.197483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:06:42.850458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191954+0000","last_change":"2023-08-11T15:44:19.156899+0000","last_active":"2023-08-11T15:44:55.191954+0000","last_peered":"2023-08-11T15:44:55.191954+0000","last_clean":"2023-08-11T15:44:55.191954+0000","last_became_active":"2023-08-11T15:44:19.156643+0000","last_became_peered":"2023-08-11T15:44:19.156643+0000","last_unstale":"2023-08-11T15:44:55.191954+0000","last_undegraded":"2023-08-11T15:44:55.191954+0000","last_fullsized":"2023-08-11T15:44:55.191954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:07.395481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190798+0000","last_change":"2023-08-11T15:44:19.857828+0000","last_active":"2023-08-11T15:44:55.190798+0000","last_peered":"2023-08-11T15:44:55.190798+0000","last_clean":"2023-08-11T15:44:55.190798+0000","last_became_active":"2023-08-11T15:44:19.856781+0000","last_became_peered":"2023-08-11T15:44:19.856781+0000","last_unstale":"2023-08-11T15:44:55.190798+0000","last_undegraded":"2023-08-11T15:44:55.190798+0000","last_fullsized":"2023-08-11T15:44:55.190798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:59:52.008608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192640+0000","last_change":"2023-08-11T15:44:24.193213+0000","last_active":"2023-08-11T15:44:55.192640+0000","last_peered":"2023-08-11T15:44:55.192640+0000","last_clean":"2023-08-11T15:44:55.192640+0000","last_became_active":"2023-08-11T15:44:24.192819+0000","last_became_peered":"2023-08-11T15:44:24.192819+0000","last_unstale":"2023-08-11T15:44:55.192640+0000","last_undegraded":"2023-08-11T15:44:55.192640+0000","last_fullsized":"2023-08-11T15:44:55.192640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:21:03.566682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'22","reported_seq":67,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:57.210608+0000","last_change":"2023-08-11T15:44:19.149753+0000","last_active":"2023-08-11T15:44:57.210608+0000","last_peered":"2023-08-11T15:44:57.210608+0000","last_clean":"2023-08-11T15:44:57.210608+0000","last_became_active":"2023-08-11T15:44:19.149420+0000","last_became_peered":"2023-08-11T15:44:19.149420+0000","last_unstale":"2023-08-11T15:44:57.210608+0000","last_undegraded":"2023-08-11T15:44:57.210608+0000","last_fullsized":"2023-08-11T15:44:57.210608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:43:16.393961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43761,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":52,"num_write":22,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195024+0000","last_change":"2023-08-11T15:44:19.861342+0000","last_active":"2023-08-11T15:44:55.195024+0000","last_peered":"2023-08-11T15:44:55.195024+0000","last_clean":"2023-08-11T15:44:55.195024+0000","last_became_active":"2023-08-11T15:44:19.860978+0000","last_became_peered":"2023-08-11T15:44:19.860978+0000","last_unstale":"2023-08-11T15:44:55.195024+0000","last_undegraded":"2023-08-11T15:44:55.195024+0000","last_fullsized":"2023-08-11T15:44:55.195024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:12.643255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191794+0000","last_change":"2023-08-11T15:44:24.191504+0000","last_active":"2023-08-11T15:44:55.191794+0000","last_peered":"2023-08-11T15:44:55.191794+0000","last_clean":"2023-08-11T15:44:55.191794+0000","last_became_active":"2023-08-11T15:44:24.191127+0000","last_became_peered":"2023-08-11T15:44:24.191127+0000","last_unstale":"2023-08-11T15:44:55.191794+0000","last_undegraded":"2023-08-11T15:44:55.191794+0000","last_fullsized":"2023-08-11T15:44:55.191794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:47:16.917614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194641+0000","last_change":"2023-08-11T15:44:19.148354+0000","last_active":"2023-08-11T15:44:55.194641+0000","last_peered":"2023-08-11T15:44:55.194641+0000","last_clean":"2023-08-11T15:44:55.194641+0000","last_became_active":"2023-08-11T15:44:19.148036+0000","last_became_peered":"2023-08-11T15:44:19.148036+0000","last_unstale":"2023-08-11T15:44:55.194641+0000","last_undegraded":"2023-08-11T15:44:55.194641+0000","last_fullsized":"2023-08-11T15:44:55.194641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:06:50.544324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195295+0000","last_change":"2023-08-11T15:44:19.862409+0000","last_active":"2023-08-11T15:44:55.195295+0000","last_peered":"2023-08-11T15:44:55.195295+0000","last_clean":"2023-08-11T15:44:55.195295+0000","last_became_active":"2023-08-11T15:44:19.861359+0000","last_became_peered":"2023-08-11T15:44:19.861359+0000","last_unstale":"2023-08-11T15:44:55.195295+0000","last_undegraded":"2023-08-11T15:44:55.195295+0000","last_fullsized":"2023-08-11T15:44:55.195295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:22:15.811283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192490+0000","last_change":"2023-08-11T15:44:24.197634+0000","last_active":"2023-08-11T15:44:55.192490+0000","last_peered":"2023-08-11T15:44:55.192490+0000","last_clean":"2023-08-11T15:44:55.192490+0000","last_became_active":"2023-08-11T15:44:24.197181+0000","last_became_peered":"2023-08-11T15:44:24.197181+0000","last_unstale":"2023-08-11T15:44:55.192490+0000","last_undegraded":"2023-08-11T15:44:55.192490+0000","last_fullsized":"2023-08-11T15:44:55.192490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:24:25.505006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190865+0000","last_change":"2023-08-11T15:44:19.147550+0000","last_active":"2023-08-11T15:44:55.190865+0000","last_peered":"2023-08-11T15:44:55.190865+0000","last_clean":"2023-08-11T15:44:55.190865+0000","last_became_active":"2023-08-11T15:44:19.147299+0000","last_became_peered":"2023-08-11T15:44:19.147299+0000","last_unstale":"2023-08-11T15:44:55.190865+0000","last_undegraded":"2023-08-11T15:44:55.190865+0000","last_fullsized":"2023-08-11T15:44:55.190865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:16.012041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194683+0000","last_change":"2023-08-11T15:44:19.862664+0000","last_active":"2023-08-11T15:44:55.194683+0000","last_peered":"2023-08-11T15:44:55.194683+0000","last_clean":"2023-08-11T15:44:55.194683+0000","last_became_active":"2023-08-11T15:44:19.862130+0000","last_became_peered":"2023-08-11T15:44:19.862130+0000","last_unstale":"2023-08-11T15:44:55.194683+0000","last_undegraded":"2023-08-11T15:44:55.194683+0000","last_fullsized":"2023-08-11T15:44:55.194683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:26:44.167213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193403+0000","last_change":"2023-08-11T15:44:24.199851+0000","last_active":"2023-08-11T15:44:55.193403+0000","last_peered":"2023-08-11T15:44:55.193403+0000","last_clean":"2023-08-11T15:44:55.193403+0000","last_became_active":"2023-08-11T15:44:24.199420+0000","last_became_peered":"2023-08-11T15:44:24.199420+0000","last_unstale":"2023-08-11T15:44:55.193403+0000","last_undegraded":"2023-08-11T15:44:55.193403+0000","last_fullsized":"2023-08-11T15:44:55.193403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:32.647518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198075+0000","last_change":"2023-08-11T15:44:19.145888+0000","last_active":"2023-08-11T15:44:55.198075+0000","last_peered":"2023-08-11T15:44:55.198075+0000","last_clean":"2023-08-11T15:44:55.198075+0000","last_became_active":"2023-08-11T15:44:19.145459+0000","last_became_peered":"2023-08-11T15:44:19.145459+0000","last_unstale":"2023-08-11T15:44:55.198075+0000","last_undegraded":"2023-08-11T15:44:55.198075+0000","last_fullsized":"2023-08-11T15:44:55.198075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:29.167474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194230+0000","last_change":"2023-08-11T15:44:19.860035+0000","last_active":"2023-08-11T15:44:55.194230+0000","last_peered":"2023-08-11T15:44:55.194230+0000","last_clean":"2023-08-11T15:44:55.194230+0000","last_became_active":"2023-08-11T15:44:19.859696+0000","last_became_peered":"2023-08-11T15:44:19.859696+0000","last_unstale":"2023-08-11T15:44:55.194230+0000","last_undegraded":"2023-08-11T15:44:55.194230+0000","last_fullsized":"2023-08-11T15:44:55.194230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:33.718669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196792+0000","last_change":"2023-08-11T15:44:24.198255+0000","last_active":"2023-08-11T15:44:55.196792+0000","last_peered":"2023-08-11T15:44:55.196792+0000","last_clean":"2023-08-11T15:44:55.196792+0000","last_became_active":"2023-08-11T15:44:24.197825+0000","last_became_peered":"2023-08-11T15:44:24.197825+0000","last_unstale":"2023-08-11T15:44:55.196792+0000","last_undegraded":"2023-08-11T15:44:55.196792+0000","last_fullsized":"2023-08-11T15:44:55.196792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:00:15.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.207182+0000","last_change":"2023-08-11T15:44:55.206745+0000","last_active":"2023-08-11T15:44:55.207182+0000","last_peered":"2023-08-11T15:44:55.207182+0000","last_clean":"2023-08-11T15:44:55.207182+0000","last_became_active":"2023-08-11T15:44:55.196900+0000","last_became_peered":"2023-08-11T15:44:55.196900+0000","last_unstale":"2023-08-11T15:44:55.207182+0000","last_undegraded":"2023-08-11T15:44:55.207182+0000","last_fullsized":"2023-08-11T15:44:55.207182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:32.529218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190529+0000","last_change":"2023-08-11T15:44:19.853273+0000","last_active":"2023-08-11T15:44:55.190529+0000","last_peered":"2023-08-11T15:44:55.190529+0000","last_clean":"2023-08-11T15:44:55.190529+0000","last_became_active":"2023-08-11T15:44:19.852914+0000","last_became_peered":"2023-08-11T15:44:19.852914+0000","last_unstale":"2023-08-11T15:44:55.190529+0000","last_undegraded":"2023-08-11T15:44:55.190529+0000","last_fullsized":"2023-08-11T15:44:55.190529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:33.263048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190457+0000","last_change":"2023-08-11T15:44:24.194429+0000","last_active":"2023-08-11T15:44:55.190457+0000","last_peered":"2023-08-11T15:44:55.190457+0000","last_clean":"2023-08-11T15:44:55.190457+0000","last_became_active":"2023-08-11T15:44:24.194089+0000","last_became_peered":"2023-08-11T15:44:24.194089+0000","last_unstale":"2023-08-11T15:44:55.190457+0000","last_undegraded":"2023-08-11T15:44:55.190457+0000","last_fullsized":"2023-08-11T15:44:55.190457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:20:34.396873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195649+0000","last_change":"2023-08-11T15:44:19.151072+0000","last_active":"2023-08-11T15:44:55.195649+0000","last_peered":"2023-08-11T15:44:55.195649+0000","last_clean":"2023-08-11T15:44:55.195649+0000","last_became_active":"2023-08-11T15:44:19.147713+0000","last_became_peered":"2023-08-11T15:44:19.147713+0000","last_unstale":"2023-08-11T15:44:55.195649+0000","last_undegraded":"2023-08-11T15:44:55.195649+0000","last_fullsized":"2023-08-11T15:44:55.195649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:43.361011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192121+0000","last_change":"2023-08-11T15:44:19.862259+0000","last_active":"2023-08-11T15:44:55.192121+0000","last_peered":"2023-08-11T15:44:55.192121+0000","last_clean":"2023-08-11T15:44:55.192121+0000","last_became_active":"2023-08-11T15:44:19.861910+0000","last_became_peered":"2023-08-11T15:44:19.861910+0000","last_unstale":"2023-08-11T15:44:55.192121+0000","last_undegraded":"2023-08-11T15:44:55.192121+0000","last_fullsized":"2023-08-11T15:44:55.192121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:31.107437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198879+0000","last_change":"2023-08-11T15:44:55.198879+0000","last_active":"2023-08-11T15:44:55.198879+0000","last_peered":"2023-08-11T15:44:55.198879+0000","last_clean":"2023-08-11T15:44:55.198879+0000","last_became_active":"2023-08-11T15:44:55.198461+0000","last_became_peered":"2023-08-11T15:44:55.198461+0000","last_unstale":"2023-08-11T15:44:55.198879+0000","last_undegraded":"2023-08-11T15:44:55.198879+0000","last_fullsized":"2023-08-11T15:44:55.198879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:04:45.789628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193224+0000","last_change":"2023-08-11T15:44:19.152608+0000","last_active":"2023-08-11T15:44:55.193224+0000","last_peered":"2023-08-11T15:44:55.193224+0000","last_clean":"2023-08-11T15:44:55.193224+0000","last_became_active":"2023-08-11T15:44:19.152255+0000","last_became_peered":"2023-08-11T15:44:19.152255+0000","last_unstale":"2023-08-11T15:44:55.193224+0000","last_undegraded":"2023-08-11T15:44:55.193224+0000","last_fullsized":"2023-08-11T15:44:55.193224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:01.248200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193585+0000","last_change":"2023-08-11T15:44:19.864003+0000","last_active":"2023-08-11T15:44:55.193585+0000","last_peered":"2023-08-11T15:44:55.193585+0000","last_clean":"2023-08-11T15:44:55.193585+0000","last_became_active":"2023-08-11T15:44:19.863661+0000","last_became_peered":"2023-08-11T15:44:19.863661+0000","last_unstale":"2023-08-11T15:44:55.193585+0000","last_undegraded":"2023-08-11T15:44:55.193585+0000","last_fullsized":"2023-08-11T15:44:55.193585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:40:30.368843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196098+0000","last_change":"2023-08-11T15:44:24.194123+0000","last_active":"2023-08-11T15:44:55.196098+0000","last_peered":"2023-08-11T15:44:55.196098+0000","last_clean":"2023-08-11T15:44:55.196098+0000","last_became_active":"2023-08-11T15:44:24.193729+0000","last_became_peered":"2023-08-11T15:44:24.193729+0000","last_unstale":"2023-08-11T15:44:55.196098+0000","last_undegraded":"2023-08-11T15:44:55.196098+0000","last_fullsized":"2023-08-11T15:44:55.196098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:46:01.331338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194389+0000","last_change":"2023-08-11T15:44:19.151367+0000","last_active":"2023-08-11T15:44:55.194389+0000","last_peered":"2023-08-11T15:44:55.194389+0000","last_clean":"2023-08-11T15:44:55.194389+0000","last_became_active":"2023-08-11T15:44:19.150991+0000","last_became_peered":"2023-08-11T15:44:19.150991+0000","last_unstale":"2023-08-11T15:44:55.194389+0000","last_undegraded":"2023-08-11T15:44:55.194389+0000","last_fullsized":"2023-08-11T15:44:55.194389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:26:13.610294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192907+0000","last_change":"2023-08-11T15:44:19.858579+0000","last_active":"2023-08-11T15:44:55.192907+0000","last_peered":"2023-08-11T15:44:55.192907+0000","last_clean":"2023-08-11T15:44:55.192907+0000","last_became_active":"2023-08-11T15:44:19.858184+0000","last_became_peered":"2023-08-11T15:44:19.858184+0000","last_unstale":"2023-08-11T15:44:55.192907+0000","last_undegraded":"2023-08-11T15:44:55.192907+0000","last_fullsized":"2023-08-11T15:44:55.192907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:51:28.014074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196948+0000","last_change":"2023-08-11T15:44:24.300526+0000","last_active":"2023-08-11T15:44:55.196948+0000","last_peered":"2023-08-11T15:44:55.196948+0000","last_clean":"2023-08-11T15:44:55.196948+0000","last_became_active":"2023-08-11T15:44:24.299883+0000","last_became_peered":"2023-08-11T15:44:24.299883+0000","last_unstale":"2023-08-11T15:44:55.196948+0000","last_undegraded":"2023-08-11T15:44:55.196948+0000","last_fullsized":"2023-08-11T15:44:55.196948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:10.792495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191233+0000","last_change":"2023-08-11T15:44:19.858116+0000","last_active":"2023-08-11T15:44:55.191233+0000","last_peered":"2023-08-11T15:44:55.191233+0000","last_clean":"2023-08-11T15:44:55.191233+0000","last_became_active":"2023-08-11T15:44:19.857728+0000","last_became_peered":"2023-08-11T15:44:19.857728+0000","last_unstale":"2023-08-11T15:44:55.191233+0000","last_undegraded":"2023-08-11T15:44:55.191233+0000","last_fullsized":"2023-08-11T15:44:55.191233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:50.561098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195613+0000","last_change":"2023-08-11T15:44:19.157904+0000","last_active":"2023-08-11T15:44:55.195613+0000","last_peered":"2023-08-11T15:44:55.195613+0000","last_clean":"2023-08-11T15:44:55.195613+0000","last_became_active":"2023-08-11T15:44:19.157497+0000","last_became_peered":"2023-08-11T15:44:19.157497+0000","last_unstale":"2023-08-11T15:44:55.195613+0000","last_undegraded":"2023-08-11T15:44:55.195613+0000","last_fullsized":"2023-08-11T15:44:55.195613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:10:52.538491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194951+0000","last_change":"2023-08-11T15:44:24.201246+0000","last_active":"2023-08-11T15:44:55.194951+0000","last_peered":"2023-08-11T15:44:55.194951+0000","last_clean":"2023-08-11T15:44:55.194951+0000","last_became_active":"2023-08-11T15:44:24.200810+0000","last_became_peered":"2023-08-11T15:44:24.200810+0000","last_unstale":"2023-08-11T15:44:55.194951+0000","last_undegraded":"2023-08-11T15:44:55.194951+0000","last_fullsized":"2023-08-11T15:44:55.194951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:41:26.333082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195176+0000","last_change":"2023-08-11T15:44:19.846296+0000","last_active":"2023-08-11T15:44:55.195176+0000","last_peered":"2023-08-11T15:44:55.195176+0000","last_clean":"2023-08-11T15:44:55.195176+0000","last_became_active":"2023-08-11T15:44:19.845890+0000","last_became_peered":"2023-08-11T15:44:19.845890+0000","last_unstale":"2023-08-11T15:44:55.195176+0000","last_undegraded":"2023-08-11T15:44:55.195176+0000","last_fullsized":"2023-08-11T15:44:55.195176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:34:40.965829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197297+0000","last_change":"2023-08-11T15:44:19.144202+0000","last_active":"2023-08-11T15:44:55.197297+0000","last_peered":"2023-08-11T15:44:55.197297+0000","last_clean":"2023-08-11T15:44:55.197297+0000","last_became_active":"2023-08-11T15:44:19.143789+0000","last_became_peered":"2023-08-11T15:44:19.143789+0000","last_unstale":"2023-08-11T15:44:55.197297+0000","last_undegraded":"2023-08-11T15:44:55.197297+0000","last_fullsized":"2023-08-11T15:44:55.197297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:15.141815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193739+0000","last_change":"2023-08-11T15:43:14.593634+0000","last_active":"2023-08-11T15:44:55.193739+0000","last_peered":"2023-08-11T15:44:55.193739+0000","last_clean":"2023-08-11T15:44:55.193739+0000","last_became_active":"2023-08-11T15:43:14.593164+0000","last_became_peered":"2023-08-11T15:43:14.593164+0000","last_unstale":"2023-08-11T15:44:55.193739+0000","last_undegraded":"2023-08-11T15:44:55.193739+0000","last_fullsized":"2023-08-11T15:44:55.193739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_clean_scrub_stamp":"2023-08-11T15:40:09.009169+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:13:50.737842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190789+0000","last_change":"2023-08-11T15:44:24.300703+0000","last_active":"2023-08-11T15:44:55.190789+0000","last_peered":"2023-08-11T15:44:55.190789+0000","last_clean":"2023-08-11T15:44:55.190789+0000","last_became_active":"2023-08-11T15:44:24.300309+0000","last_became_peered":"2023-08-11T15:44:24.300309+0000","last_unstale":"2023-08-11T15:44:55.190789+0000","last_undegraded":"2023-08-11T15:44:55.190789+0000","last_fullsized":"2023-08-11T15:44:55.190789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:11.965214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201619+0000","last_change":"2023-08-11T15:44:55.201619+0000","last_active":"2023-08-11T15:44:55.201619+0000","last_peered":"2023-08-11T15:44:55.201619+0000","last_clean":"2023-08-11T15:44:55.201619+0000","last_became_active":"2023-08-11T15:44:55.201233+0000","last_became_peered":"2023-08-11T15:44:55.201233+0000","last_unstale":"2023-08-11T15:44:55.201619+0000","last_undegraded":"2023-08-11T15:44:55.201619+0000","last_fullsized":"2023-08-11T15:44:55.201619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:11:25.230480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193945+0000","last_change":"2023-08-11T15:44:19.148113+0000","last_active":"2023-08-11T15:44:55.193945+0000","last_peered":"2023-08-11T15:44:55.193945+0000","last_clean":"2023-08-11T15:44:55.193945+0000","last_became_active":"2023-08-11T15:44:19.147793+0000","last_became_peered":"2023-08-11T15:44:19.147793+0000","last_unstale":"2023-08-11T15:44:55.193945+0000","last_undegraded":"2023-08-11T15:44:55.193945+0000","last_fullsized":"2023-08-11T15:44:55.193945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:28.804554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191891+0000","last_change":"2023-08-11T15:44:24.194418+0000","last_active":"2023-08-11T15:44:55.191891+0000","last_peered":"2023-08-11T15:44:55.191891+0000","last_clean":"2023-08-11T15:44:55.191891+0000","last_became_active":"2023-08-11T15:44:24.193943+0000","last_became_peered":"2023-08-11T15:44:24.193943+0000","last_unstale":"2023-08-11T15:44:55.191891+0000","last_undegraded":"2023-08-11T15:44:55.191891+0000","last_fullsized":"2023-08-11T15:44:55.191891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:56:38.790282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195211+0000","last_change":"2023-08-11T15:44:19.151476+0000","last_active":"2023-08-11T15:44:55.195211+0000","last_peered":"2023-08-11T15:44:55.195211+0000","last_clean":"2023-08-11T15:44:55.195211+0000","last_became_active":"2023-08-11T15:44:19.151140+0000","last_became_peered":"2023-08-11T15:44:19.151140+0000","last_unstale":"2023-08-11T15:44:55.195211+0000","last_undegraded":"2023-08-11T15:44:55.195211+0000","last_fullsized":"2023-08-11T15:44:55.195211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:43.957395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196271+0000","last_change":"2023-08-11T15:44:19.862255+0000","last_active":"2023-08-11T15:44:55.196271+0000","last_peered":"2023-08-11T15:44:55.196271+0000","last_clean":"2023-08-11T15:44:55.196271+0000","last_became_active":"2023-08-11T15:44:19.861907+0000","last_became_peered":"2023-08-11T15:44:19.861907+0000","last_unstale":"2023-08-11T15:44:55.196271+0000","last_undegraded":"2023-08-11T15:44:55.196271+0000","last_fullsized":"2023-08-11T15:44:55.196271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:33:45.120706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194419+0000","last_change":"2023-08-11T15:44:24.300513+0000","last_active":"2023-08-11T15:44:55.194419+0000","last_peered":"2023-08-11T15:44:55.194419+0000","last_clean":"2023-08-11T15:44:55.194419+0000","last_became_active":"2023-08-11T15:44:24.300074+0000","last_became_peered":"2023-08-11T15:44:24.300074+0000","last_unstale":"2023-08-11T15:44:55.194419+0000","last_undegraded":"2023-08-11T15:44:55.194419+0000","last_fullsized":"2023-08-11T15:44:55.194419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:22.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193528+0000","last_change":"2023-08-11T15:44:19.153794+0000","last_active":"2023-08-11T15:44:55.193528+0000","last_peered":"2023-08-11T15:44:55.193528+0000","last_clean":"2023-08-11T15:44:55.193528+0000","last_became_active":"2023-08-11T15:44:19.153392+0000","last_became_peered":"2023-08-11T15:44:19.153392+0000","last_unstale":"2023-08-11T15:44:55.193528+0000","last_undegraded":"2023-08-11T15:44:55.193528+0000","last_fullsized":"2023-08-11T15:44:55.193528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:59:18.150146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197135+0000","last_change":"2023-08-11T15:44:19.859799+0000","last_active":"2023-08-11T15:44:55.197135+0000","last_peered":"2023-08-11T15:44:55.197135+0000","last_clean":"2023-08-11T15:44:55.197135+0000","last_became_active":"2023-08-11T15:44:19.859411+0000","last_became_peered":"2023-08-11T15:44:19.859411+0000","last_unstale":"2023-08-11T15:44:55.197135+0000","last_undegraded":"2023-08-11T15:44:55.197135+0000","last_fullsized":"2023-08-11T15:44:55.197135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:50:04.069525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194744+0000","last_change":"2023-08-11T15:44:24.190272+0000","last_active":"2023-08-11T15:44:55.194744+0000","last_peered":"2023-08-11T15:44:55.194744+0000","last_clean":"2023-08-11T15:44:55.194744+0000","last_became_active":"2023-08-11T15:44:24.189849+0000","last_became_peered":"2023-08-11T15:44:24.189849+0000","last_unstale":"2023-08-11T15:44:55.194744+0000","last_undegraded":"2023-08-11T15:44:55.194744+0000","last_fullsized":"2023-08-11T15:44:55.194744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:21:45.389663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198407+0000","last_change":"2023-08-11T15:44:19.154985+0000","last_active":"2023-08-11T15:44:55.198407+0000","last_peered":"2023-08-11T15:44:55.198407+0000","last_clean":"2023-08-11T15:44:55.198407+0000","last_became_active":"2023-08-11T15:44:19.154694+0000","last_became_peered":"2023-08-11T15:44:19.154694+0000","last_unstale":"2023-08-11T15:44:55.198407+0000","last_undegraded":"2023-08-11T15:44:55.198407+0000","last_fullsized":"2023-08-11T15:44:55.198407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:22:34.297160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190129+0000","last_change":"2023-08-11T15:44:19.852247+0000","last_active":"2023-08-11T15:44:55.190129+0000","last_peered":"2023-08-11T15:44:55.190129+0000","last_clean":"2023-08-11T15:44:55.190129+0000","last_became_active":"2023-08-11T15:44:19.851456+0000","last_became_peered":"2023-08-11T15:44:19.851456+0000","last_unstale":"2023-08-11T15:44:55.190129+0000","last_undegraded":"2023-08-11T15:44:55.190129+0000","last_fullsized":"2023-08-11T15:44:55.190129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:43:25.512545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.199076+0000","last_change":"2023-08-11T15:44:24.300440+0000","last_active":"2023-08-11T15:44:55.199076+0000","last_peered":"2023-08-11T15:44:55.199076+0000","last_clean":"2023-08-11T15:44:55.199076+0000","last_became_active":"2023-08-11T15:44:24.299981+0000","last_became_peered":"2023-08-11T15:44:24.299981+0000","last_unstale":"2023-08-11T15:44:55.199076+0000","last_undegraded":"2023-08-11T15:44:55.199076+0000","last_fullsized":"2023-08-11T15:44:55.199076+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:10:48.146770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190574+0000","last_change":"2023-08-11T15:44:19.151015+0000","last_active":"2023-08-11T15:44:55.190574+0000","last_peered":"2023-08-11T15:44:55.190574+0000","last_clean":"2023-08-11T15:44:55.190574+0000","last_became_active":"2023-08-11T15:44:19.150677+0000","last_became_peered":"2023-08-11T15:44:19.150677+0000","last_unstale":"2023-08-11T15:44:55.190574+0000","last_undegraded":"2023-08-11T15:44:55.190574+0000","last_fullsized":"2023-08-11T15:44:55.190574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:44.956238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194239+0000","last_change":"2023-08-11T15:44:19.861033+0000","last_active":"2023-08-11T15:44:55.194239+0000","last_peered":"2023-08-11T15:44:55.194239+0000","last_clean":"2023-08-11T15:44:55.194239+0000","last_became_active":"2023-08-11T15:44:19.860643+0000","last_became_peered":"2023-08-11T15:44:19.860643+0000","last_unstale":"2023-08-11T15:44:55.194239+0000","last_undegraded":"2023-08-11T15:44:55.194239+0000","last_fullsized":"2023-08-11T15:44:55.194239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:32:25.731474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193496+0000","last_change":"2023-08-11T15:44:24.199389+0000","last_active":"2023-08-11T15:44:55.193496+0000","last_peered":"2023-08-11T15:44:55.193496+0000","last_clean":"2023-08-11T15:44:55.193496+0000","last_became_active":"2023-08-11T15:44:24.196226+0000","last_became_peered":"2023-08-11T15:44:24.196226+0000","last_unstale":"2023-08-11T15:44:55.193496+0000","last_undegraded":"2023-08-11T15:44:55.193496+0000","last_fullsized":"2023-08-11T15:44:55.193496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:05:34.885169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194914+0000","last_change":"2023-08-11T15:44:19.152895+0000","last_active":"2023-08-11T15:44:55.194914+0000","last_peered":"2023-08-11T15:44:55.194914+0000","last_clean":"2023-08-11T15:44:55.194914+0000","last_became_active":"2023-08-11T15:44:19.152577+0000","last_became_peered":"2023-08-11T15:44:19.152577+0000","last_unstale":"2023-08-11T15:44:55.194914+0000","last_undegraded":"2023-08-11T15:44:55.194914+0000","last_fullsized":"2023-08-11T15:44:55.194914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:23:00.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192632+0000","last_change":"2023-08-11T15:44:19.861101+0000","last_active":"2023-08-11T15:44:55.192632+0000","last_peered":"2023-08-11T15:44:55.192632+0000","last_clean":"2023-08-11T15:44:55.192632+0000","last_became_active":"2023-08-11T15:44:19.860529+0000","last_became_peered":"2023-08-11T15:44:19.860529+0000","last_unstale":"2023-08-11T15:44:55.192632+0000","last_undegraded":"2023-08-11T15:44:55.192632+0000","last_fullsized":"2023-08-11T15:44:55.192632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:30:49.161446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197970+0000","last_change":"2023-08-11T15:44:24.202856+0000","last_active":"2023-08-11T15:44:55.197970+0000","last_peered":"2023-08-11T15:44:55.197970+0000","last_clean":"2023-08-11T15:44:55.197970+0000","last_became_active":"2023-08-11T15:44:24.202317+0000","last_became_peered":"2023-08-11T15:44:24.202317+0000","last_unstale":"2023-08-11T15:44:55.197970+0000","last_undegraded":"2023-08-11T15:44:55.197970+0000","last_fullsized":"2023-08-11T15:44:55.197970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:41:10.081275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195594+0000","last_change":"2023-08-11T15:44:19.148711+0000","last_active":"2023-08-11T15:44:55.195594+0000","last_peered":"2023-08-11T15:44:55.195594+0000","last_clean":"2023-08-11T15:44:55.195594+0000","last_became_active":"2023-08-11T15:44:19.148361+0000","last_became_peered":"2023-08-11T15:44:19.148361+0000","last_unstale":"2023-08-11T15:44:55.195594+0000","last_undegraded":"2023-08-11T15:44:55.195594+0000","last_fullsized":"2023-08-11T15:44:55.195594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:05:10.590173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195941+0000","last_change":"2023-08-11T15:44:19.859562+0000","last_active":"2023-08-11T15:44:55.195941+0000","last_peered":"2023-08-11T15:44:55.195941+0000","last_clean":"2023-08-11T15:44:55.195941+0000","last_became_active":"2023-08-11T15:44:19.859243+0000","last_became_peered":"2023-08-11T15:44:19.859243+0000","last_unstale":"2023-08-11T15:44:55.195941+0000","last_undegraded":"2023-08-11T15:44:55.195941+0000","last_fullsized":"2023-08-11T15:44:55.195941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:50:37.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197923+0000","last_change":"2023-08-11T15:44:24.300317+0000","last_active":"2023-08-11T15:44:55.197923+0000","last_peered":"2023-08-11T15:44:55.197923+0000","last_clean":"2023-08-11T15:44:55.197923+0000","last_became_active":"2023-08-11T15:44:24.299841+0000","last_became_peered":"2023-08-11T15:44:24.299841+0000","last_unstale":"2023-08-11T15:44:55.197923+0000","last_undegraded":"2023-08-11T15:44:55.197923+0000","last_fullsized":"2023-08-11T15:44:55.197923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:39.799187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194316+0000","last_change":"2023-08-11T15:44:19.149111+0000","last_active":"2023-08-11T15:44:55.194316+0000","last_peered":"2023-08-11T15:44:55.194316+0000","last_clean":"2023-08-11T15:44:55.194316+0000","last_became_active":"2023-08-11T15:44:19.148173+0000","last_became_peered":"2023-08-11T15:44:19.148173+0000","last_unstale":"2023-08-11T15:44:55.194316+0000","last_undegraded":"2023-08-11T15:44:55.194316+0000","last_fullsized":"2023-08-11T15:44:55.194316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:13:23.513921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194513+0000","last_change":"2023-08-11T15:44:19.863724+0000","last_active":"2023-08-11T15:44:55.194513+0000","last_peered":"2023-08-11T15:44:55.194513+0000","last_clean":"2023-08-11T15:44:55.194513+0000","last_became_active":"2023-08-11T15:44:19.863268+0000","last_became_peered":"2023-08-11T15:44:19.863268+0000","last_unstale":"2023-08-11T15:44:55.194513+0000","last_undegraded":"2023-08-11T15:44:55.194513+0000","last_fullsized":"2023-08-11T15:44:55.194513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:59.448457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190468+0000","last_change":"2023-08-11T15:44:24.191301+0000","last_active":"2023-08-11T15:44:55.190468+0000","last_peered":"2023-08-11T15:44:55.190468+0000","last_clean":"2023-08-11T15:44:55.190468+0000","last_became_active":"2023-08-11T15:44:24.190686+0000","last_became_peered":"2023-08-11T15:44:24.190686+0000","last_unstale":"2023-08-11T15:44:55.190468+0000","last_undegraded":"2023-08-11T15:44:55.190468+0000","last_fullsized":"2023-08-11T15:44:55.190468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:26:07.780061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195770+0000","last_change":"2023-08-11T15:44:19.147756+0000","last_active":"2023-08-11T15:44:55.195770+0000","last_peered":"2023-08-11T15:44:55.195770+0000","last_clean":"2023-08-11T15:44:55.195770+0000","last_became_active":"2023-08-11T15:44:19.147426+0000","last_became_peered":"2023-08-11T15:44:19.147426+0000","last_unstale":"2023-08-11T15:44:55.195770+0000","last_undegraded":"2023-08-11T15:44:55.195770+0000","last_fullsized":"2023-08-11T15:44:55.195770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:17:56.074888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195206+0000","last_change":"2023-08-11T15:44:19.863354+0000","last_active":"2023-08-11T15:44:55.195206+0000","last_peered":"2023-08-11T15:44:55.195206+0000","last_clean":"2023-08-11T15:44:55.195206+0000","last_became_active":"2023-08-11T15:44:19.862700+0000","last_became_peered":"2023-08-11T15:44:19.862700+0000","last_unstale":"2023-08-11T15:44:55.195206+0000","last_undegraded":"2023-08-11T15:44:55.195206+0000","last_fullsized":"2023-08-11T15:44:55.195206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:17:30.462182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189633+0000","last_change":"2023-08-11T15:44:24.204657+0000","last_active":"2023-08-11T15:44:55.189633+0000","last_peered":"2023-08-11T15:44:55.189633+0000","last_clean":"2023-08-11T15:44:55.189633+0000","last_became_active":"2023-08-11T15:44:24.204108+0000","last_became_peered":"2023-08-11T15:44:24.204108+0000","last_unstale":"2023-08-11T15:44:55.189633+0000","last_undegraded":"2023-08-11T15:44:55.189633+0000","last_fullsized":"2023-08-11T15:44:55.189633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:25.563864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191508+0000","last_change":"2023-08-11T15:44:19.147014+0000","last_active":"2023-08-11T15:44:55.191508+0000","last_peered":"2023-08-11T15:44:55.191508+0000","last_clean":"2023-08-11T15:44:55.191508+0000","last_became_active":"2023-08-11T15:44:19.146720+0000","last_became_peered":"2023-08-11T15:44:19.146720+0000","last_unstale":"2023-08-11T15:44:55.191508+0000","last_undegraded":"2023-08-11T15:44:55.191508+0000","last_fullsized":"2023-08-11T15:44:55.191508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:44:30.278828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201693+0000","last_change":"2023-08-11T15:44:55.201693+0000","last_active":"2023-08-11T15:44:55.201693+0000","last_peered":"2023-08-11T15:44:55.201693+0000","last_clean":"2023-08-11T15:44:55.201693+0000","last_became_active":"2023-08-11T15:44:55.201360+0000","last_became_peered":"2023-08-11T15:44:55.201360+0000","last_unstale":"2023-08-11T15:44:55.201693+0000","last_undegraded":"2023-08-11T15:44:55.201693+0000","last_fullsized":"2023-08-11T15:44:55.201693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:54:43.667678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200415+0000","last_change":"2023-08-11T15:44:55.197220+0000","last_active":"2023-08-11T15:44:55.200415+0000","last_peered":"2023-08-11T15:44:55.200415+0000","last_clean":"2023-08-11T15:44:55.200415+0000","last_became_active":"2023-08-11T15:44:55.196808+0000","last_became_peered":"2023-08-11T15:44:55.196808+0000","last_unstale":"2023-08-11T15:44:55.200415+0000","last_undegraded":"2023-08-11T15:44:55.200415+0000","last_fullsized":"2023-08-11T15:44:55.200415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:42:42.724543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196765+0000","last_change":"2023-08-11T15:44:19.158301+0000","last_active":"2023-08-11T15:44:55.196765+0000","last_peered":"2023-08-11T15:44:55.196765+0000","last_clean":"2023-08-11T15:44:55.196765+0000","last_became_active":"2023-08-11T15:44:19.157990+0000","last_became_peered":"2023-08-11T15:44:19.157990+0000","last_unstale":"2023-08-11T15:44:55.196765+0000","last_undegraded":"2023-08-11T15:44:55.196765+0000","last_fullsized":"2023-08-11T15:44:55.196765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:47:15.846659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195216+0000","last_change":"2023-08-11T15:44:19.857357+0000","last_active":"2023-08-11T15:44:55.195216+0000","last_peered":"2023-08-11T15:44:55.195216+0000","last_clean":"2023-08-11T15:44:55.195216+0000","last_became_active":"2023-08-11T15:44:19.857020+0000","last_became_peered":"2023-08-11T15:44:19.857020+0000","last_unstale":"2023-08-11T15:44:55.195216+0000","last_undegraded":"2023-08-11T15:44:55.195216+0000","last_fullsized":"2023-08-11T15:44:55.195216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:00.529783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192767+0000","last_change":"2023-08-11T15:44:19.863314+0000","last_active":"2023-08-11T15:44:55.192767+0000","last_peered":"2023-08-11T15:44:55.192767+0000","last_clean":"2023-08-11T15:44:55.192767+0000","last_became_active":"2023-08-11T15:44:19.862964+0000","last_became_peered":"2023-08-11T15:44:19.862964+0000","last_unstale":"2023-08-11T15:44:55.192767+0000","last_undegraded":"2023-08-11T15:44:55.192767+0000","last_fullsized":"2023-08-11T15:44:55.192767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:45:00.154066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195297+0000","last_change":"2023-08-11T15:44:19.850575+0000","last_active":"2023-08-11T15:44:55.195297+0000","last_peered":"2023-08-11T15:44:55.195297+0000","last_clean":"2023-08-11T15:44:55.195297+0000","last_became_active":"2023-08-11T15:44:19.850231+0000","last_became_peered":"2023-08-11T15:44:19.850231+0000","last_unstale":"2023-08-11T15:44:55.195297+0000","last_undegraded":"2023-08-11T15:44:55.195297+0000","last_fullsized":"2023-08-11T15:44:55.195297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:37.529897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195464+0000","last_change":"2023-08-11T15:44:19.857387+0000","last_active":"2023-08-11T15:44:55.195464+0000","last_peered":"2023-08-11T15:44:55.195464+0000","last_clean":"2023-08-11T15:44:55.195464+0000","last_became_active":"2023-08-11T15:44:19.856830+0000","last_became_peered":"2023-08-11T15:44:19.856830+0000","last_unstale":"2023-08-11T15:44:55.195464+0000","last_undegraded":"2023-08-11T15:44:55.195464+0000","last_fullsized":"2023-08-11T15:44:55.195464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:18:48.525612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189975+0000","last_change":"2023-08-11T15:44:19.868112+0000","last_active":"2023-08-11T15:44:55.189975+0000","last_peered":"2023-08-11T15:44:55.189975+0000","last_clean":"2023-08-11T15:44:55.189975+0000","last_became_active":"2023-08-11T15:44:19.867610+0000","last_became_peered":"2023-08-11T15:44:19.867610+0000","last_unstale":"2023-08-11T15:44:55.189975+0000","last_undegraded":"2023-08-11T15:44:55.189975+0000","last_fullsized":"2023-08-11T15:44:55.189975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:13:03.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196422+0000","last_change":"2023-08-11T15:44:19.858349+0000","last_active":"2023-08-11T15:44:55.196422+0000","last_peered":"2023-08-11T15:44:55.196422+0000","last_clean":"2023-08-11T15:44:55.196422+0000","last_became_active":"2023-08-11T15:44:19.858041+0000","last_became_peered":"2023-08-11T15:44:19.858041+0000","last_unstale":"2023-08-11T15:44:55.196422+0000","last_undegraded":"2023-08-11T15:44:55.196422+0000","last_fullsized":"2023-08-11T15:44:55.196422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:52.727062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192538+0000","last_change":"2023-08-11T15:44:19.865383+0000","last_active":"2023-08-11T15:44:55.192538+0000","last_peered":"2023-08-11T15:44:55.192538+0000","last_clean":"2023-08-11T15:44:55.192538+0000","last_became_active":"2023-08-11T15:44:19.865065+0000","last_became_peered":"2023-08-11T15:44:19.865065+0000","last_unstale":"2023-08-11T15:44:55.192538+0000","last_undegraded":"2023-08-11T15:44:55.192538+0000","last_fullsized":"2023-08-11T15:44:55.192538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:28:38.454441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195185+0000","last_change":"2023-08-11T15:44:19.859297+0000","last_active":"2023-08-11T15:44:55.195185+0000","last_peered":"2023-08-11T15:44:55.195185+0000","last_clean":"2023-08-11T15:44:55.195185+0000","last_became_active":"2023-08-11T15:44:19.857517+0000","last_became_peered":"2023-08-11T15:44:19.857517+0000","last_unstale":"2023-08-11T15:44:55.195185+0000","last_undegraded":"2023-08-11T15:44:55.195185+0000","last_fullsized":"2023-08-11T15:44:55.195185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:03:57.202219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201849+0000","last_change":"2023-08-11T15:44:55.201849+0000","last_active":"2023-08-11T15:44:55.201849+0000","last_peered":"2023-08-11T15:44:55.201849+0000","last_clean":"2023-08-11T15:44:55.201849+0000","last_became_active":"2023-08-11T15:44:55.201480+0000","last_became_peered":"2023-08-11T15:44:55.201480+0000","last_unstale":"2023-08-11T15:44:55.201849+0000","last_undegraded":"2023-08-11T15:44:55.201849+0000","last_fullsized":"2023-08-11T15:44:55.201849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:57.682087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190486+0000","last_change":"2023-08-11T15:44:19.861953+0000","last_active":"2023-08-11T15:44:55.190486+0000","last_peered":"2023-08-11T15:44:55.190486+0000","last_clean":"2023-08-11T15:44:55.190486+0000","last_became_active":"2023-08-11T15:44:19.861523+0000","last_became_peered":"2023-08-11T15:44:19.861523+0000","last_unstale":"2023-08-11T15:44:55.190486+0000","last_undegraded":"2023-08-11T15:44:55.190486+0000","last_fullsized":"2023-08-11T15:44:55.190486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:22:05.760149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194869+0000","last_change":"2023-08-11T15:44:19.849283+0000","last_active":"2023-08-11T15:44:55.194869+0000","last_peered":"2023-08-11T15:44:55.194869+0000","last_clean":"2023-08-11T15:44:55.194869+0000","last_became_active":"2023-08-11T15:44:19.848304+0000","last_became_peered":"2023-08-11T15:44:19.848304+0000","last_unstale":"2023-08-11T15:44:55.194869+0000","last_undegraded":"2023-08-11T15:44:55.194869+0000","last_fullsized":"2023-08-11T15:44:55.194869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:03.496778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190770+0000","last_change":"2023-08-11T15:44:19.853532+0000","last_active":"2023-08-11T15:44:55.190770+0000","last_peered":"2023-08-11T15:44:55.190770+0000","last_clean":"2023-08-11T15:44:55.190770+0000","last_became_active":"2023-08-11T15:44:19.853182+0000","last_became_peered":"2023-08-11T15:44:19.853182+0000","last_unstale":"2023-08-11T15:44:55.190770+0000","last_undegraded":"2023-08-11T15:44:55.190770+0000","last_fullsized":"2023-08-11T15:44:55.190770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:39:40.398308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195608+0000","last_change":"2023-08-11T15:44:19.860571+0000","last_active":"2023-08-11T15:44:55.195608+0000","last_peered":"2023-08-11T15:44:55.195608+0000","last_clean":"2023-08-11T15:44:55.195608+0000","last_became_active":"2023-08-11T15:44:19.858570+0000","last_became_peered":"2023-08-11T15:44:19.858570+0000","last_unstale":"2023-08-11T15:44:55.195608+0000","last_undegraded":"2023-08-11T15:44:55.195608+0000","last_fullsized":"2023-08-11T15:44:55.195608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:26:39.509633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192077+0000","last_change":"2023-08-11T15:44:19.865988+0000","last_active":"2023-08-11T15:44:55.192077+0000","last_peered":"2023-08-11T15:44:55.192077+0000","last_clean":"2023-08-11T15:44:55.192077+0000","last_became_active":"2023-08-11T15:44:19.865613+0000","last_became_peered":"2023-08-11T15:44:19.865613+0000","last_unstale":"2023-08-11T15:44:55.192077+0000","last_undegraded":"2023-08-11T15:44:55.192077+0000","last_fullsized":"2023-08-11T15:44:55.192077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:46:16.828564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44559,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":25,"num_read_kb":58,"num_write":61,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":133677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":490303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.77400000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1120199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.629}]},{"osd":6,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.73699999999999999}]},{"osd":8,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.871}]},{"osd":9,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.89300000000000002}]},{"osd":11,"last update":"Fri Aug 11 15:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.75},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.879}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":494981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.748},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.60699999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.82399999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.98499999999999999}]},{"osd":6,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.88},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.90300000000000002}]},{"osd":8,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.97399999999999998}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.014}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":490576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.75800000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72499999999999998,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.60399999999999998}]},{"osd":1,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.746},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.85799999999999998,"5min":1.05,"15min":1.05},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.435,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99199999999999999,"5min":1.077,"15min":1.077},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.92600000000000005}]},{"osd":5,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":1.05}]},{"osd":7,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95399999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.91300000000000003}]},{"osd":9,"last update":"Fri Aug 11 15:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.069},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.90000000000000002,"15min":0.91500000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.083,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":1.081}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.97799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1085349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.45500000000000002,"15min":0.44800000000000001},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.504}]},{"osd":2,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.436}]},{"osd":3,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626}]},{"osd":5,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.749},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.80500000000000005}]},{"osd":8,"last update":"Fri Aug 11 15:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.79300000000000004,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.498,"15min":0.498},"max":{"1min":4.8890000000000002,"5min":4.8890000000000002,"15min":4.8890000000000002},"last":0.76700000000000002}]},{"osd":9,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.59299999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.94199999999999995}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":486207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":2.077,"15min":2.077},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.008,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.581,"15min":1.581},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92500000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.7,"15min":1.7},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0269999999999999,"5min":1.22,"15min":1.22},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1319999999999999,"5min":1.623,"15min":1.623},"last":0.78500000000000003}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.61799999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.63,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.877}]},{"osd":8,"last update":"Fri Aug 11 15:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.90000000000000002,"15min":0.89200000000000002},"min":{"1min":0.64500000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.014}]},{"osd":9,"last update":"Fri Aug 11 15:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96199999999999997,"15min":0.97399999999999998},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.121,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.98999999999999999,"15min":1},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1559999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":0.61699999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":529968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.76600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88600000000000001,"5min":4.9020000000000001,"15min":4.9020000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.45900000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80800000000000005,"5min":1.593,"15min":1.593},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.024,"15min":1.024},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.435},{"interface":"front","average":{"1min":0.501,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74099999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.754},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":1.05,"15min":1.05},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.65800000000000003}]},{"osd":8,"last update":"Fri Aug 11 15:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67700000000000005,"15min":0.66600000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60499999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1180000000000001,"5min":1.298,"15min":1.298},"last":0.68999999999999995}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.622}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":486297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70099999999999996,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.874,"5min":6.0609999999999999,"15min":6.0609999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64200000000000002,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88400000000000001,"5min":6.101,"15min":6.101},"last":0.502}]},{"osd":2,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.621,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.999,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.441,"15min":0.433},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46100000000000002,"15min":0.44700000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Aug 11 15:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.65300000000000002,"15min":0.61299999999999999},"min":{"1min":0.56000000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.54300000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.056,"5min":1.175,"15min":1.175},"last":1.034}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.032,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.92500000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.74399999999999999,"15min":0.72799999999999998},"min":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.87},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.78900000000000003,"15min":0.77300000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.89000000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1779999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.149,"5min":1.276,"15min":1.276},"last":1.145}]},{"osd":9,"last update":"Fri Aug 11 15:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97799999999999998,"5min":1.339,"15min":1.339},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.98499999999999999,"15min":1.0129999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.194,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.0169999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.113}]},{"osd":11,"last update":"Fri Aug 11 15:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.095}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":530002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.379}]},{"osd":1,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54600000000000004,"15min":0.53300000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80300000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72399999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Aug 11 15:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.44},"min":{"1min":0.29699999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.434},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40300000000000002}]},{"osd":6,"last update":"Fri Aug 11 15:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Aug 11 15:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51300000000000001}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65200000000000002}]},{"osd":10,"last update":"Fri Aug 11 15:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.755}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":486311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.47699999999999998,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68100000000000005,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.878},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Aug 11 15:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.55600000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94399999999999995,"5min":1.105,"15min":1.105},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81000000000000005}]},{"osd":5,"last update":"Fri Aug 11 15:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.53800000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.72399999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.58799999999999997,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.97199999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.624}]},{"osd":9,"last update":"Fri Aug 11 15:44:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95099999999999996,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.266,"5min":5.266,"15min":5.266},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri Aug 11 15:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.498}]},{"osd":5,"last update":"Fri Aug 11 15:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83599999999999997,"5min":1.552,"15min":1.552},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":1.502,"15min":1.502},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.75600000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.016,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.56000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93899999999999995,"5min":1.526,"15min":1.526},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Aug 11 15:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.63200000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.874},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1080522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79500000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60599999999999998,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83299999999999996,"5min":5.359,"15min":5.359},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.434,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.872,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77000000000000002,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.749}]},{"osd":2,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.36899999999999999},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41999999999999998,"15min":0.40400000000000003},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.73099999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.032,"15min":1.032},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82599999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.626,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.78200000000000003}]},{"osd":8,"last update":"Fri Aug 11 15:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.874}]},{"osd":10,"last update":"Fri Aug 11 15:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.83699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":43795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":43955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:45:01.266 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-11T15:45:01.267 INFO:tasks.cephadm:Setting up client nodes... 2023-08-11T15:45:01.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-08-11T15:45:01.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:01 smithi134 ceph-mon[130447]: pgmap v293: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.7 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:01 smithi032 ceph-mon[97103]: pgmap v293: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.7 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:01.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:01 smithi192 ceph-mon[130806]: pgmap v293: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.7 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:02.028 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:02.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:02 smithi134 ceph-mon[130447]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:02.546 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:02 smithi032 ceph-mon[97103]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:02 smithi192 ceph-mon[130806]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:03.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:03 smithi134 ceph-mon[130447]: pgmap v294: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.9 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 -- 172.21.15.32:0/3977177477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384102580 msgr2=0x7ff384108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/3977177477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384102580 0x7ff384108e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b0d0 tx=0x7ff37c00b3e0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 -- 172.21.15.32:0/3977177477 shutdown_connections 2023-08-11T15:45:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/3977177477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384102580 0x7ff384108e30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/3977177477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3841009e0 0x7ff384100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/3977177477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384101be0 0x7ff384102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 -- 172.21.15.32:0/3977177477 >> 172.21.15.32:0/3977177477 conn(0x7ff3840fbfa0 msgr2=0x7ff3840fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.542+0000 7ff38b9c0700 1 -- 172.21.15.32:0/3977177477 shutdown_connections 2023-08-11T15:45:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.543+0000 7ff38b9c0700 1 -- 172.21.15.32:0/3977177477 wait complete. 2023-08-11T15:45:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 Processor -- start 2023-08-11T15:45:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 -- start start 2023-08-11T15:45:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3841009e0 0x7ff384196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384101be0 0x7ff384196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102580 0x7ff38419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38419e2f0 con 0x7ff384101be0 2023-08-11T15:45:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38419e430 con 0x7ff3841009e0 2023-08-11T15:45:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38b9c0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38419e5c0 con 0x7ff384102580 2023-08-11T15:45:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff388f5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384101be0 0x7ff384196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff388f5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384101be0 0x7ff384196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47166/0 (socket says 172.21.15.32:47166) 2023-08-11T15:45:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff388f5b700 1 -- 172.21.15.32:0/1339505412 learned_addr learned my addr 172.21.15.32:0/1339505412 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.544+0000 7ff38975c700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3841009e0 0x7ff384196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff389f5d700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102580 0x7ff38419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff38975c700 1 -- 172.21.15.32:0/1339505412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102580 msgr2=0x7ff38419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff38975c700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102580 0x7ff38419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff38975c700 1 -- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384101be0 msgr2=0x7ff384196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff38975c700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384101be0 0x7ff384196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff38975c700 1 -- 172.21.15.32:0/1339505412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c009d00 con 0x7ff3841009e0 2023-08-11T15:45:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff388f5b700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384101be0 0x7ff384196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff38975c700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3841009e0 0x7ff384196760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e9c0 tx=0x7ff38000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.545+0000 7ff37a7fc700 1 -- 172.21.15.32:0/1339505412 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380004220 con 0x7ff3841009e0 2023-08-11T15:45:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.546+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38419e8a0 con 0x7ff3841009e0 2023-08-11T15:45:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.546+0000 7ff37a7fc700 1 -- 172.21.15.32:0/1339505412 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff380004380 con 0x7ff3841009e0 2023-08-11T15:45:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.546+0000 7ff37a7fc700 1 -- 172.21.15.32:0/1339505412 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380003c50 con 0x7ff3841009e0 2023-08-11T15:45:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.546+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38419eda0 con 0x7ff3841009e0 2023-08-11T15:45:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.547+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38410c1f0 con 0x7ff3841009e0 2023-08-11T15:45:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.549+0000 7ff37a7fc700 1 -- 172.21.15.32:0/1339505412 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff38000c7c0 con 0x7ff3841009e0 2023-08-11T15:45:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.550+0000 7ff37a7fc700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff37009ffe0 0x7ff3700a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.550+0000 7ff37a7fc700 1 -- 172.21.15.32:0/1339505412 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff380014070 con 0x7ff3841009e0 2023-08-11T15:45:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.550+0000 7ff388f5b700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff37009ffe0 0x7ff3700a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:03.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:03 smithi032 ceph-mon[97103]: pgmap v294: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.9 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.551+0000 7ff388f5b700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff37009ffe0 0x7ff3700a24a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3740104e0 tx=0x7ff374007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.552+0000 7ff37a7fc700 1 -- 172.21.15.32:0/1339505412 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff3800952f0 con 0x7ff3841009e0 2023-08-11T15:45:03.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:03 smithi192 ceph-mon[130806]: pgmap v294: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.9 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.841+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff38410c480 con 0x7ff3841009e0 2023-08-11T15:45:03.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.850+0000 7ff37a7fc700 1 -- 172.21.15.32:0/1339505412 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff380098910 con 0x7ff3841009e0 2023-08-11T15:45:03.853 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-08-11T15:45:03.853 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB/V9Zkk3hYMhAAeqDioCMCIbZXXlAGbEFNEw== 2023-08-11T15:45:03.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff37009ffe0 msgr2=0x7ff3700a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:03.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff37009ffe0 0x7ff3700a24a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3740104e0 tx=0x7ff374007480 comp rx=0 tx=0).stop 2023-08-11T15:45:03.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3841009e0 msgr2=0x7ff384196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:03.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3841009e0 0x7ff384196760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e9c0 tx=0x7ff38000ecd0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 shutdown_connections 2023-08-11T15:45:03.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff37009ffe0 0x7ff3700a24a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102580 0x7ff38419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3841009e0 0x7ff384196760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 --2- 172.21.15.32:0/1339505412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff384101be0 0x7ff384196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:03.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 >> 172.21.15.32:0/1339505412 conn(0x7ff3840fbfa0 msgr2=0x7ff3840fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:03.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.852+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 shutdown_connections 2023-08-11T15:45:03.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:03.853+0000 7ff38b9c0700 1 -- 172.21.15.32:0/1339505412 wait complete. 2023-08-11T15:45:04.326 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:45:04.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-08-11T15:45:04.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-11T15:45:04.433 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-11T15:45:04.435 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-08-11T15:45:04.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph mgr dump --format=json 2023-08-11T15:45:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:04 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1339505412' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T15:45:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:04 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T15:45:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:04 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-11T15:45:04.535 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:04 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1339505412' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T15:45:04.535 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:04 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T15:45:04.536 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:04 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-11T15:45:04.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:04 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1339505412' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T15:45:04.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:04 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T15:45:04.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:04 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-11T15:45:05.202 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:05.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:05 smithi134 ceph-mon[130447]: pgmap v295: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:05.502 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:05 smithi032 ceph-mon[97103]: pgmap v295: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:05.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:05 smithi192 ceph-mon[130806]: pgmap v295: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 -- 172.21.15.32:0/4274280066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0101300 msgr2=0x7f81a010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 --2- 172.21.15.32:0/4274280066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0101300 0x7f81a010d810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f819800b0a0 tx=0x7f819800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 -- 172.21.15.32:0/4274280066 shutdown_connections 2023-08-11T15:45:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 --2- 172.21.15.32:0/4274280066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0101300 0x7f81a010d810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 --2- 172.21.15.32:0/4274280066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0106940 0x7f81a0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 --2- 172.21.15.32:0/4274280066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81a0100940 0x7f81a0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 -- 172.21.15.32:0/4274280066 >> 172.21.15.32:0/4274280066 conn(0x7f81a00fc410 msgr2=0x7f81a00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 -- 172.21.15.32:0/4274280066 shutdown_connections 2023-08-11T15:45:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.658+0000 7f81a73ca700 1 -- 172.21.15.32:0/4274280066 wait complete. 2023-08-11T15:45:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.659+0000 7f81a73ca700 1 Processor -- start 2023-08-11T15:45:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.659+0000 7f81a73ca700 1 -- start start 2023-08-11T15:45:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a73ca700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81a0100940 0x7f81a0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a73ca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0101300 0x7f81a0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a73ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0106940 0x7f81a00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a73ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01a4fb0 con 0x7f81a0106940 2023-08-11T15:45:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a73ca700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01a5120 con 0x7f81a0101300 2023-08-11T15:45:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a73ca700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01a52e0 con 0x7f81a0100940 2023-08-11T15:45:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a5967700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0106940 0x7f81a00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a5166700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81a0100940 0x7f81a0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a5166700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81a0100940 0x7f81a0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46868/0 (socket says 172.21.15.32:46868) 2023-08-11T15:45:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a4965700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0101300 0x7f81a0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a5166700 1 -- 172.21.15.32:0/1810894313 learned_addr learned my addr 172.21.15.32:0/1810894313 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a5967700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0106940 0x7f81a00736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50572/0 (socket says 172.21.15.32:50572) 2023-08-11T15:45:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.660+0000 7f81a5967700 1 -- 172.21.15.32:0/1810894313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81a0100940 msgr2=0x7f81a0072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81a5967700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81a0100940 0x7f81a0072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81a5967700 1 -- 172.21.15.32:0/1810894313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0101300 msgr2=0x7f81a0073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81a5967700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0101300 0x7f81a0073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81a5967700 1 -- 172.21.15.32:0/1810894313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198009d00 con 0x7f81a0106940 2023-08-11T15:45:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81a5967700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0106940 0x7f81a00736e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f819800b0a0 tx=0x7f8198007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81967fc700 1 -- 172.21.15.32:0/1810894313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819800e040 con 0x7f81a0106940 2023-08-11T15:45:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81967fc700 1 -- 172.21.15.32:0/1810894313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f819801d470 con 0x7f81a0106940 2023-08-11T15:45:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81967fc700 1 -- 172.21.15.32:0/1810894313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81980052a0 con 0x7f81a0106940 2023-08-11T15:45:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a01a5560 con 0x7f81a0106940 2023-08-11T15:45:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.661+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a01a5980 con 0x7f81a0106940 2023-08-11T15:45:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.662+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a004e620 con 0x7f81a0106940 2023-08-11T15:45:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.664+0000 7f81967fc700 1 -- 172.21.15.32:0/1810894313 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f819801d5e0 con 0x7f81a0106940 2023-08-11T15:45:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.666+0000 7f81967fc700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f818c09ffe0 0x7f818c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.666+0000 7f81967fc700 1 -- 172.21.15.32:0/1810894313 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81980ccc00 con 0x7f81a0106940 2023-08-11T15:45:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.666+0000 7f81a5166700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f818c09ffe0 0x7f818c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.666+0000 7f81a5166700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f818c09ffe0 0x7f818c0a24a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f819c009c80 tx=0x7f819c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.670+0000 7f81967fc700 1 -- 172.21.15.32:0/1810894313 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8198096550 con 0x7f81a0106940 2023-08-11T15:45:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.941+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f81a0110b80 con 0x7f81a0106940 2023-08-11T15:45:06.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.947+0000 7f81967fc700 1 -- 172.21.15.32:0/1810894313 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+258101 (secure 0 0 0) 0x7f81980b9020 con 0x7f81a0106940 2023-08-11T15:45:06.953 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:06.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f818c09ffe0 msgr2=0x7f818c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f818c09ffe0 0x7f818c0a24a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f819c009c80 tx=0x7f819c009260 comp rx=0 tx=0).stop 2023-08-11T15:45:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0106940 msgr2=0x7f81a00736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0106940 0x7f81a00736e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f819800b0a0 tx=0x7f8198007990 comp rx=0 tx=0).stop 2023-08-11T15:45:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 shutdown_connections 2023-08-11T15:45:06.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f818c09ffe0 0x7f818c0a24a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a0106940 0x7f81a00736e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81a0100940 0x7f81a0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 --2- 172.21.15.32:0/1810894313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a0101300 0x7f81a0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 >> 172.21.15.32:0/1810894313 conn(0x7f81a00fc410 msgr2=0x7f81a00fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 shutdown_connections 2023-08-11T15:45:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:06.952+0000 7f81a73ca700 1 -- 172.21.15.32:0/1810894313 wait complete. 2023-08-11T15:45:07.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:07 smithi032 ceph-mon[97103]: pgmap v296: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:07.439 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:07 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1810894313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-11T15:45:07.440 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2482310260},{"type":"v1","addr":"172.21.15.32:6801","nonce":2482310260}]},"active_addr":"172.21.15.32:6801/2482310260","active_change":"2023-08-11T15:37:53.986695+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.32:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2111930192}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2323942786}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2001360921}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1349164373}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1382306322}]}]} 2023-08-11T15:45:07.447 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-08-11T15:45:07.447 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-08-11T15:45:07.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd dump --format=json 2023-08-11T15:45:07.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:07 smithi192 ceph-mon[130806]: pgmap v296: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:07.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:07 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1810894313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-11T15:45:07.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:07 smithi134 ceph-mon[130447]: pgmap v296: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:07.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:07 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1810894313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-11T15:45:08.189 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:09.476 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:09 smithi032 ceph-mon[97103]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.4 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:09.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:09 smithi192 ceph-mon[130806]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.4 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:09.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:09 smithi134 ceph-mon[130447]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.4 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-11T15:45:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 -- 172.21.15.32:0/830007354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c106940 msgr2=0x7f690c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 --2- 172.21.15.32:0/830007354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c106940 0x7f690c106d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009990 tx=0x7f68fc009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 -- 172.21.15.32:0/830007354 shutdown_connections 2023-08-11T15:45:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 --2- 172.21.15.32:0/830007354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c106940 0x7f690c106d20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 --2- 172.21.15.32:0/830007354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f690c101300 0x7f690c10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 --2- 172.21.15.32:0/830007354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690c100940 0x7f690c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 -- 172.21.15.32:0/830007354 >> 172.21.15.32:0/830007354 conn(0x7f690c0fc410 msgr2=0x7f690c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 -- 172.21.15.32:0/830007354 shutdown_connections 2023-08-11T15:45:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.734+0000 7f69115c9700 1 -- 172.21.15.32:0/830007354 wait complete. 2023-08-11T15:45:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.735+0000 7f69115c9700 1 Processor -- start 2023-08-11T15:45:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.735+0000 7f69115c9700 1 -- start start 2023-08-11T15:45:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f69115c9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690c100940 0x7f690c072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f69115c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c101300 0x7f690c073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f69115c9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f690c106940 0x7f690c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f69115c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1a4ee0 con 0x7f690c101300 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f69115c9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1a5050 con 0x7f690c106940 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f69115c9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1a5210 con 0x7f690c100940 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f690a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c101300 0x7f690c073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f690a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c101300 0x7f690c073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50600/0 (socket says 172.21.15.32:50600) 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f690a7fc700 1 -- 172.21.15.32:0/3408837772 learned_addr learned my addr 172.21.15.32:0/3408837772 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f690affd700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690c100940 0x7f690c072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f690b7fe700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f690c106940 0x7f690c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f690a7fc700 1 -- 172.21.15.32:0/3408837772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690c100940 msgr2=0x7f690c072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.736+0000 7f690a7fc700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690c100940 0x7f690c072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f690a7fc700 1 -- 172.21.15.32:0/3408837772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f690c106940 msgr2=0x7f690c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f690a7fc700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f690c106940 0x7f690c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f690a7fc700 1 -- 172.21.15.32:0/3408837772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0095f0 con 0x7f690c101300 2023-08-11T15:45:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f690a7fc700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c101300 0x7f690c073160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f690000e9a0 tx=0x7f690000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f68f3fff700 1 -- 172.21.15.32:0/3408837772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900009790 con 0x7f690c101300 2023-08-11T15:45:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c1a54f0 con 0x7f690c101300 2023-08-11T15:45:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f68f3fff700 1 -- 172.21.15.32:0/3408837772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6900010430 con 0x7f690c101300 2023-08-11T15:45:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f68f3fff700 1 -- 172.21.15.32:0/3408837772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900018680 con 0x7f690c101300 2023-08-11T15:45:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.737+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c1a59f0 con 0x7f690c101300 2023-08-11T15:45:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.738+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c04e620 con 0x7f690c101300 2023-08-11T15:45:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.740+0000 7f68f3fff700 1 -- 172.21.15.32:0/3408837772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f69000105a0 con 0x7f690c101300 2023-08-11T15:45:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.741+0000 7f68f3fff700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f68f409ffe0 0x7f68f40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.742+0000 7f68f3fff700 1 -- 172.21.15.32:0/3408837772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6900014070 con 0x7f690c101300 2023-08-11T15:45:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.742+0000 7f690affd700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f68f409ffe0 0x7f68f40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.742+0000 7f690affd700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f68f409ffe0 0x7f68f40a24a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00b2e0 tx=0x7f68fc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.746+0000 7f68f3fff700 1 -- 172.21.15.32:0/3408837772 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6900051c30 con 0x7f690c101300 2023-08-11T15:45:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.952+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f690c110a60 con 0x7f690c101300 2023-08-11T15:45:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.954+0000 7f68f3fff700 1 -- 172.21.15.32:0/3408837772 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f69000951c0 con 0x7f690c101300 2023-08-11T15:45:09.956 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:09.957 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:55.185701+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:45:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f68f409ffe0 msgr2=0x7f68f40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f68f409ffe0 0x7f68f40a24a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00b2e0 tx=0x7f68fc005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c101300 msgr2=0x7f690c073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c101300 0x7f690c073160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f690000e9a0 tx=0x7f690000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 shutdown_connections 2023-08-11T15:45:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f68f409ffe0 0x7f68f40a24a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f690c106940 0x7f690c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690c100940 0x7f690c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 --2- 172.21.15.32:0/3408837772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f690c101300 0x7f690c073160 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.956+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 >> 172.21.15.32:0/3408837772 conn(0x7f690c0fc410 msgr2=0x7f690c0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.957+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 shutdown_connections 2023-08-11T15:45:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:09.957+0000 7f69115c9700 1 -- 172.21.15.32:0/3408837772 wait complete. 2023-08-11T15:45:10.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:10 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3408837772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:10.484 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-08-11T15:45:10.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd dump --format=json 2023-08-11T15:45:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:10 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3408837772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:10.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:10 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3408837772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:11.220 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:11.524 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:11 smithi032 ceph-mon[97103]: pgmap v298: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-08-11T15:45:11.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:11 smithi192 ceph-mon[130806]: pgmap v298: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-08-11T15:45:11.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:11 smithi134 ceph-mon[130447]: pgmap v298: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-08-11T15:45:12.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 -- 172.21.15.32:0/3206089918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3908073670 msgr2=0x7f390810cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:12.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 --2- 172.21.15.32:0/3206089918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3908073670 0x7f390810cc40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009990 tx=0x7f38f0009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 -- 172.21.15.32:0/3206089918 shutdown_connections 2023-08-11T15:45:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 --2- 172.21.15.32:0/3206089918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f390810d180 0x7f390810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 --2- 172.21.15.32:0/3206089918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 0x7f3908073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 --2- 172.21.15.32:0/3206089918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3908073670 0x7f390810cc40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 -- 172.21.15.32:0/3206089918 >> 172.21.15.32:0/3206089918 conn(0x7f39080fc410 msgr2=0x7f39080fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 -- 172.21.15.32:0/3206089918 shutdown_connections 2023-08-11T15:45:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.782+0000 7f390cb49700 1 -- 172.21.15.32:0/3206089918 wait complete. 2023-08-11T15:45:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.783+0000 7f390cb49700 1 Processor -- start 2023-08-11T15:45:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.783+0000 7f390cb49700 1 -- start start 2023-08-11T15:45:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f390cb49700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 0x7f390819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f390cb49700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3908073670 0x7f390819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:12.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f390cb49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390810d180 0x7f3908198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:12.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f390cb49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39081990d0 con 0x7f390810d180 2023-08-11T15:45:12.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f390cb49700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908199240 con 0x7f3908073670 2023-08-11T15:45:12.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f390cb49700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908199400 con 0x7f3908072d50 2023-08-11T15:45:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f38fe59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 0x7f390819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f38fe59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 0x7f390819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46892/0 (socket says 172.21.15.32:46892) 2023-08-11T15:45:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f38fe59c700 1 -- 172.21.15.32:0/450588655 learned_addr learned my addr 172.21.15.32:0/450588655 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f390659c700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3908073670 0x7f390819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.784+0000 7f3906d9d700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390810d180 0x7f3908198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fe59c700 1 -- 172.21.15.32:0/450588655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3908073670 msgr2=0x7f390819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fe59c700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3908073670 0x7f390819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fe59c700 1 -- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390810d180 msgr2=0x7f3908198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fe59c700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390810d180 0x7f3908198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fe59c700 1 -- 172.21.15.32:0/450588655 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009c40 con 0x7f3908072d50 2023-08-11T15:45:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f390659c700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3908073670 0x7f390819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fe59c700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 0x7f390819e9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9a0 tx=0x7f38f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fffff700 1 -- 172.21.15.32:0/450588655 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8009790 con 0x7f3908072d50 2023-08-11T15:45:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fffff700 1 -- 172.21.15.32:0/450588655 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38f800ce80 con 0x7f3908072d50 2023-08-11T15:45:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f00095f0 con 0x7f3908072d50 2023-08-11T15:45:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.785+0000 7f38fffff700 1 -- 172.21.15.32:0/450588655 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8018680 con 0x7f3908072d50 2023-08-11T15:45:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.786+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39081a5800 con 0x7f3908072d50 2023-08-11T15:45:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.787+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390804e620 con 0x7f3908072d50 2023-08-11T15:45:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.789+0000 7f38fffff700 1 -- 172.21.15.32:0/450588655 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f38f8010430 con 0x7f3908072d50 2023-08-11T15:45:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.790+0000 7f38fffff700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f38f409ffe0 0x7f38f40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.790+0000 7f38fffff700 1 -- 172.21.15.32:0/450588655 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f38f8014070 con 0x7f3908072d50 2023-08-11T15:45:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.790+0000 7f390659c700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f38f409ffe0 0x7f38f40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.791+0000 7f390659c700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f38f409ffe0 0x7f38f40a24a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b2e0 tx=0x7f38f0005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:12.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:12.794+0000 7f38fffff700 1 -- 172.21.15.32:0/450588655 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f38f8051c30 con 0x7f3908072d50 2023-08-11T15:45:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.006+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3908112bd0 con 0x7f3908072d50 2023-08-11T15:45:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.008+0000 7f38fffff700 1 -- 172.21.15.32:0/450588655 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f38f8098cd0 con 0x7f3908072d50 2023-08-11T15:45:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:13.011 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:55.185701+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:45:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f38f409ffe0 msgr2=0x7f38f40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f38f409ffe0 0x7f38f40a24a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b2e0 tx=0x7f38f0005960 comp rx=0 tx=0).stop 2023-08-11T15:45:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 msgr2=0x7f390819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 0x7f390819e9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9a0 tx=0x7f38f800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 shutdown_connections 2023-08-11T15:45:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f38f409ffe0 0x7f38f40a24a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f390810d180 0x7f3908198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3908072d50 0x7f390819e9e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 --2- 172.21.15.32:0/450588655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3908073670 0x7f390819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 >> 172.21.15.32:0/450588655 conn(0x7f39080fc410 msgr2=0x7f390810f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 shutdown_connections 2023-08-11T15:45:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:13.011+0000 7f390cb49700 1 -- 172.21.15.32:0/450588655 wait complete. 2023-08-11T15:45:13.202 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:12 smithi032 ceph-mon[97103]: pgmap v299: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-08-11T15:45:13.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:12 smithi134 ceph-mon[130447]: pgmap v299: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-08-11T15:45:13.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:12 smithi192 ceph-mon[130806]: pgmap v299: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-08-11T15:45:13.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-08-11T15:45:13.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-08-11T15:45:13.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-08-11T15:45:13.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-08-11T15:45:13.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-08-11T15:45:13.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-08-11T15:45:13.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-08-11T15:45:13.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-08-11T15:45:13.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-08-11T15:45:13.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-08-11T15:45:13.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-08-11T15:45:13.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-08-11T15:45:14.061 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:13 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/450588655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:14.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:13 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/450588655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:14.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:13 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/450588655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:15.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:14 smithi134 ceph-mon[130447]: pgmap v300: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-08-11T15:45:15.274 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:14 smithi032 ceph-mon[97103]: pgmap v300: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-08-11T15:45:15.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:14 smithi192 ceph-mon[130806]: pgmap v300: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-08-11T15:45:15.391 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:17.274 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:17 smithi032 ceph-mon[97103]: pgmap v301: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:17.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:17 smithi134 ceph-mon[130447]: pgmap v301: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:17.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:17 smithi192 ceph-mon[130806]: pgmap v301: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:19.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:19 smithi032 ceph-mon[97103]: pgmap v302: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:19.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:19 smithi134 ceph-mon[130447]: pgmap v302: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:19.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:19 smithi192 ceph-mon[130806]: pgmap v302: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:20.404 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:20 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:45:20.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:20 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:45:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:20 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:45:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 -- 172.21.15.32:0/2867530535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 msgr2=0x7fd17010a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 --2- 172.21.15.32:0/2867530535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 0x7fd17010a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd160009990 tx=0x7fd160009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 -- 172.21.15.32:0/2867530535 shutdown_connections 2023-08-11T15:45:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 --2- 172.21.15.32:0/2867530535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170101990 0x7fd170101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 --2- 172.21.15.32:0/2867530535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd17010ad70 0x7fd17010d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 --2- 172.21.15.32:0/2867530535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 0x7fd17010a7a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:20.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 -- 172.21.15.32:0/2867530535 >> 172.21.15.32:0/2867530535 conn(0x7fd1700fb1c0 msgr2=0x7fd1700fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:20.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 -- 172.21.15.32:0/2867530535 shutdown_connections 2023-08-11T15:45:20.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.347+0000 7fd176628700 1 -- 172.21.15.32:0/2867530535 wait complete. 2023-08-11T15:45:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.348+0000 7fd176628700 1 Processor -- start 2023-08-11T15:45:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.349+0000 7fd176628700 1 -- start start 2023-08-11T15:45:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.349+0000 7fd176628700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170101990 0x7fd17019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.349+0000 7fd176628700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 0x7fd17019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.349+0000 7fd176628700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd17010ad70 0x7fd1701990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.349+0000 7fd176628700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701997c0 con 0x7fd170101990 2023-08-11T15:45:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.349+0000 7fd176628700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170199930 con 0x7fd17010ad70 2023-08-11T15:45:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.349+0000 7fd176628700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170199af0 con 0x7fd1701022b0 2023-08-11T15:45:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd175626700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170101990 0x7fd17019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 0x7fd17019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 0x7fd17019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34404/0 (socket says 172.21.15.32:34404) 2023-08-11T15:45:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd175626700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170101990 0x7fd17019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42814/0 (socket says 172.21.15.32:42814) 2023-08-11T15:45:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 -- 172.21.15.32:0/4262549530 learned_addr learned my addr 172.21.15.32:0/4262549530 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd175e27700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd17010ad70 0x7fd1701990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 -- 172.21.15.32:0/4262549530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd17010ad70 msgr2=0x7fd1701990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd17010ad70 0x7fd1701990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 -- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170101990 msgr2=0x7fd17019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170101990 0x7fd17019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 -- 172.21.15.32:0/4262549530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1600095f0 con 0x7fd1701022b0 2023-08-11T15:45:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd175626700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170101990 0x7fd17019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.350+0000 7fd174e25700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 0x7fd17019ee50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd160004730 tx=0x7fd160004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.351+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16001d070 con 0x7fd1701022b0 2023-08-11T15:45:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.351+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd170199d70 con 0x7fd1701022b0 2023-08-11T15:45:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.351+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1600047d0 con 0x7fd1701022b0 2023-08-11T15:45:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.351+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600053b0 con 0x7fd1701022b0 2023-08-11T15:45:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.351+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1701a5970 con 0x7fd1701022b0 2023-08-11T15:45:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1700fff20 con 0x7fd1701022b0 2023-08-11T15:45:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.355+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd160004940 con 0x7fd1701022b0 2023-08-11T15:45:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.357+0000 7fd1667fc700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd15c0a0060 0x7fd15c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.357+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1600cd0c0 con 0x7fd1701022b0 2023-08-11T15:45:20.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.357+0000 7fd175626700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd15c0a0060 0x7fd15c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.357+0000 7fd1667fc700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] conn(0x7fd15c0ab080 0x7fd15c0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.357+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 --> [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd15c0adbf0 con 0x7fd15c0ab080 2023-08-11T15:45:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.357+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1600cd430 con 0x7fd1701022b0 2023-08-11T15:45:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.358+0000 7fd175e27700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] conn(0x7fd15c0ab080 0x7fd15c0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.358+0000 7fd175626700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd15c0a0060 0x7fd15c0a2520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005950 tx=0x7fd16c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.358+0000 7fd175e27700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] conn(0x7fd15c0ab080 0x7fd15c0ad520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.360+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== osd.0 v2:172.21.15.32:6802/99848982 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd15c0adbf0 con 0x7fd15c0ab080 2023-08-11T15:45:20.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.380+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 --> [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd17010c150 con 0x7fd15c0ab080 2023-08-11T15:45:20.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd1667fc700 1 -- 172.21.15.32:0/4262549530 <== osd.0 v2:172.21.15.32:6802/99848982 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd17010c150 con 0x7fd15c0ab080 2023-08-11T15:45:20.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] conn(0x7fd15c0ab080 msgr2=0x7fd15c0ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:20.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] conn(0x7fd15c0ab080 0x7fd15c0ad520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:20.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd15c0a0060 msgr2=0x7fd15c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:20.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd15c0a0060 0x7fd15c0a2520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005950 tx=0x7fd16c00b410 comp rx=0 tx=0).stop 2023-08-11T15:45:20.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 msgr2=0x7fd17019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:20.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 --2- 172.21.15.32:0/4262549530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1701022b0 0x7fd17019ee50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd160004730 tx=0x7fd160004760 comp rx=0 tx=0).stop 2023-08-11T15:45:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd175e27700 1 -- 172.21.15.32:0/4262549530 reap_dead start 2023-08-11T15:45:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 shutdown_connections 2023-08-11T15:45:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 >> 172.21.15.32:0/4262549530 conn(0x7fd1700fb1c0 msgr2=0x7fd17010b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 shutdown_connections 2023-08-11T15:45:20.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.381+0000 7fd176628700 1 -- 172.21.15.32:0/4262549530 wait complete. 2023-08-11T15:45:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:21.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.318+0000 7f359f2fe700 1 -- 172.21.15.32:0/3398498634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35981037a0 msgr2=0x7f35981077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.318+0000 7f359f2fe700 1 --2- 172.21.15.32:0/3398498634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35981037a0 0x7f35981077f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3588009990 tx=0x7f3588009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.319+0000 7f359f2fe700 1 -- 172.21.15.32:0/3398498634 shutdown_connections 2023-08-11T15:45:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.319+0000 7f359f2fe700 1 --2- 172.21.15.32:0/3398498634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3598102df0 0x7f35981031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.319+0000 7f359f2fe700 1 --2- 172.21.15.32:0/3398498634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3598107ed0 0x7f359810faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.319+0000 7f359f2fe700 1 --2- 172.21.15.32:0/3398498634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35981037a0 0x7f35981077f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.319+0000 7f359f2fe700 1 -- 172.21.15.32:0/3398498634 >> 172.21.15.32:0/3398498634 conn(0x7f35980fe660 msgr2=0x7f3598100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.319+0000 7f359f2fe700 1 -- 172.21.15.32:0/3398498634 shutdown_connections 2023-08-11T15:45:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.319+0000 7f359f2fe700 1 -- 172.21.15.32:0/3398498634 wait complete. 2023-08-11T15:45:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 Processor -- start 2023-08-11T15:45:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 -- start start 2023-08-11T15:45:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3598102df0 0x7f359819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35981037a0 0x7f359819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3598107ed0 0x7f3598198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3598199190 con 0x7f3598107ed0 2023-08-11T15:45:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3598199300 con 0x7f3598102df0 2023-08-11T15:45:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359f2fe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981994c0 con 0x7f35981037a0 2023-08-11T15:45:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.320+0000 7f359d09a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3598102df0 0x7f359819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359c899700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35981037a0 0x7f359819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3598102df0 0x7f359819eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:60332/0 (socket says 172.21.15.32:60332) 2023-08-11T15:45:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 -- 172.21.15.32:0/2559808976 learned_addr learned my addr 172.21.15.32:0/2559808976 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359c899700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35981037a0 0x7f359819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34386/0 (socket says 172.21.15.32:34386) 2023-08-11T15:45:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d89b700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3598107ed0 0x7f3598198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 -- 172.21.15.32:0/2559808976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35981037a0 msgr2=0x7f359819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:21.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35981037a0 0x7f359819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 -- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3598107ed0 msgr2=0x7f3598198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3598107ed0 0x7f3598198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 -- 172.21.15.32:0/2559808976 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880095f0 con 0x7f3598102df0 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.321+0000 7f359d09a700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3598102df0 0x7f359819eab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f359400e9a0 tx=0x7f359400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.322+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594009790 con 0x7f3598102df0 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.322+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3594010430 con 0x7f3598102df0 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.322+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35981997a0 con 0x7f3598102df0 2023-08-11T15:45:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.322+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594018680 con 0x7f3598102df0 2023-08-11T15:45:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.322+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35981a5aa0 con 0x7f3598102df0 2023-08-11T15:45:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.323+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3598112d80 con 0x7f3598102df0 2023-08-11T15:45:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.326+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f35940105a0 con 0x7f3598102df0 2023-08-11T15:45:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.328+0000 7f358e7fc700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35840a0060 0x7f35840a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.328+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3594014070 con 0x7f3598102df0 2023-08-11T15:45:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.328+0000 7f359c899700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35840a0060 0x7f35840a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.328+0000 7f358e7fc700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] conn(0x7f35840ab080 0x7f35840ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.328+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 --> [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35840adbf0 con 0x7f35840ab080 2023-08-11T15:45:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.328+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35940cbb20 con 0x7f3598102df0 2023-08-11T15:45:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.329+0000 7f359d89b700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] conn(0x7f35840ab080 0x7f35840ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.329+0000 7f359c899700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35840a0060 0x7f35840a2520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3588006010 tx=0x7f3588005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.329+0000 7f359d89b700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] conn(0x7f35840ab080 0x7f35840ad520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.331+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== osd.3 v2:172.21.15.32:6810/862642324 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f35840adbf0 con 0x7f35840ab080 2023-08-11T15:45:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.351+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 --> [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f359804e620 con 0x7f35840ab080 2023-08-11T15:45:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.351+0000 7f358e7fc700 1 -- 172.21.15.32:0/2559808976 <== osd.3 v2:172.21.15.32:6810/862642324 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f359804e620 con 0x7f35840ab080 2023-08-11T15:45:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] conn(0x7f35840ab080 msgr2=0x7f35840ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] conn(0x7f35840ab080 0x7f35840ad520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35840a0060 msgr2=0x7f35840a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35840a0060 0x7f35840a2520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3588006010 tx=0x7f3588005960 comp rx=0 tx=0).stop 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3598102df0 msgr2=0x7f359819eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 --2- 172.21.15.32:0/2559808976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3598102df0 0x7f359819eab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f359400e9a0 tx=0x7f359400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359d89b700 1 -- 172.21.15.32:0/2559808976 reap_dead start 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 shutdown_connections 2023-08-11T15:45:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 >> 172.21.15.32:0/2559808976 conn(0x7f35980fe660 msgr2=0x7f35980ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:21.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 shutdown_connections 2023-08-11T15:45:21.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:20.352+0000 7f359f2fe700 1 -- 172.21.15.32:0/2559808976 wait complete. 2023-08-11T15:45:21.178 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:21 smithi032 ceph-mon[97103]: pgmap v303: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:45:21.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:21 smithi134 ceph-mon[130447]: pgmap v303: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:45:21.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:21 smithi192 ceph-mon[130806]: pgmap v303: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:45:22.119 INFO:teuthology.orchestra.run.smithi032.stdout:34359738441 2023-08-11T15:45:22.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-08-11T15:45:22.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.548+0000 7fec2883f700 1 -- 172.21.15.32:0/2934818888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec24100940 msgr2=0x7fec24100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.548+0000 7fec2883f700 1 --2- 172.21.15.32:0/2934818888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec24100940 0x7fec24100dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec18009990 tx=0x7fec18009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.549+0000 7fec2883f700 1 -- 172.21.15.32:0/2934818888 shutdown_connections 2023-08-11T15:45:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.549+0000 7fec2883f700 1 --2- 172.21.15.32:0/2934818888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24101300 0x7fec2410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.549+0000 7fec2883f700 1 --2- 172.21.15.32:0/2934818888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec24106940 0x7fec24106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.549+0000 7fec2883f700 1 --2- 172.21.15.32:0/2934818888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec24100940 0x7fec24100dc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.549+0000 7fec2883f700 1 -- 172.21.15.32:0/2934818888 >> 172.21.15.32:0/2934818888 conn(0x7fec240fc410 msgr2=0x7fec240fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.549+0000 7fec2883f700 1 -- 172.21.15.32:0/2934818888 shutdown_connections 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.549+0000 7fec2883f700 1 -- 172.21.15.32:0/2934818888 wait complete. 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 Processor -- start 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 -- start start 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24100940 0x7fec2419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec24101300 0x7fec2419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec24106940 0x7fec24198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec24199230 con 0x7fec24100940 2023-08-11T15:45:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec241993a0 con 0x7fec24101300 2023-08-11T15:45:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.550+0000 7fec2883f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec24199560 con 0x7fec24106940 2023-08-11T15:45:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24100940 0x7fec2419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24100940 0x7fec2419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42836/0 (socket says 172.21.15.32:42836) 2023-08-11T15:45:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 -- 172.21.15.32:0/3068513294 learned_addr learned my addr 172.21.15.32:0/3068513294 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec21d9b700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec24101300 0x7fec2419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec22d9d700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec24106940 0x7fec24198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 -- 172.21.15.32:0/3068513294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec24106940 msgr2=0x7fec24198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec24106940 0x7fec24198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 -- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec24101300 msgr2=0x7fec2419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec24101300 0x7fec2419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 -- 172.21.15.32:0/3068513294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec180095f0 con 0x7fec24100940 2023-08-11T15:45:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec21d9b700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec24101300 0x7fec2419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.551+0000 7fec2259c700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24100940 0x7fec2419ea90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e9a0 tx=0x7fec1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.552+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14009790 con 0x7fec24100940 2023-08-11T15:45:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.552+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec14010430 con 0x7fec24100940 2023-08-11T15:45:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.552+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14018680 con 0x7fec24100940 2023-08-11T15:45:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.552+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec24199840 con 0x7fec24100940 2023-08-11T15:45:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.552+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec241a5a10 con 0x7fec24100940 2023-08-11T15:45:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.553+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec2404e620 con 0x7fec24100940 2023-08-11T15:45:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.555+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fec140105a0 con 0x7fec24100940 2023-08-11T15:45:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.557+0000 7fec137fe700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec0c09ffe0 0x7fec0c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.557+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fec14014070 con 0x7fec24100940 2023-08-11T15:45:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.557+0000 7fec21d9b700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec0c09ffe0 0x7fec0c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.557+0000 7fec137fe700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] conn(0x7fec0c0ab000 0x7fec0c0ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:22.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.557+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 --> [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec0c0adb70 con 0x7fec0c0ab000 2023-08-11T15:45:22.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.557+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec140cbed0 con 0x7fec24100940 2023-08-11T15:45:22.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.558+0000 7fec22d9d700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] conn(0x7fec0c0ab000 0x7fec0c0ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:22.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.558+0000 7fec21d9b700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec0c09ffe0 0x7fec0c0a24a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec2419a2b0 tx=0x7fec18005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:22.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.558+0000 7fec22d9d700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] conn(0x7fec0c0ab000 0x7fec0c0ad4a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:22.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.559+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== osd.10 v2:172.21.15.134:6824/1937504104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fec0c0adb70 con 0x7fec0c0ab000 2023-08-11T15:45:22.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.577+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 --> [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec24110c20 con 0x7fec0c0ab000 2023-08-11T15:45:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.577+0000 7fec137fe700 1 -- 172.21.15.32:0/3068513294 <== osd.10 v2:172.21.15.134:6824/1937504104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fec1c00f040 con 0x7fec0c0ab000 2023-08-11T15:45:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] conn(0x7fec0c0ab000 msgr2=0x7fec0c0ad4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] conn(0x7fec0c0ab000 0x7fec0c0ad4a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec0c09ffe0 msgr2=0x7fec0c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fec0c09ffe0 0x7fec0c0a24a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec2419a2b0 tx=0x7fec18005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24100940 msgr2=0x7fec2419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 --2- 172.21.15.32:0/3068513294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24100940 0x7fec2419ea90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e9a0 tx=0x7fec1400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec22d9d700 1 -- 172.21.15.32:0/3068513294 reap_dead start 2023-08-11T15:45:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 shutdown_connections 2023-08-11T15:45:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 >> 172.21.15.32:0/3068513294 conn(0x7fec240fc410 msgr2=0x7fec2410bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 shutdown_connections 2023-08-11T15:45:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:22.578+0000 7fec2883f700 1 -- 172.21.15.32:0/3068513294 wait complete. 2023-08-11T15:45:22.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:22 smithi032 ceph-mon[97103]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:45:22.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:22.856 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:22 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:23.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:22 smithi134 ceph-mon[130447]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:45:23.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:23.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:22 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:23.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:22 smithi192 ceph-mon[130806]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:45:23.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:23.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:22 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 -- 172.21.15.32:0/979971675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6801058e0 msgr2=0x7fc680103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 --2- 172.21.15.32:0/979971675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6801058e0 0x7fc680103370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc670009990 tx=0x7fc670009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 -- 172.21.15.32:0/979971675 shutdown_connections 2023-08-11T15:45:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 --2- 172.21.15.32:0/979971675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc680068440 0x7fc6800688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 --2- 172.21.15.32:0/979971675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6801058e0 0x7fc680103370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 --2- 172.21.15.32:0/979971675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6801038b0 0x7fc680103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 -- 172.21.15.32:0/979971675 >> 172.21.15.32:0/979971675 conn(0x7fc680076920 msgr2=0x7fc680076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 -- 172.21.15.32:0/979971675 shutdown_connections 2023-08-11T15:45:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.334+0000 7fc6867bf700 1 -- 172.21.15.32:0/979971675 wait complete. 2023-08-11T15:45:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.335+0000 7fc6867bf700 1 Processor -- start 2023-08-11T15:45:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.335+0000 7fc6867bf700 1 -- start start 2023-08-11T15:45:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc6867bf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc680068440 0x7fc68019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc6867bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6801038b0 0x7fc68019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc6867bf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6801058e0 0x7fc6801a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc6867bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6801a1ff0 con 0x7fc6801038b0 2023-08-11T15:45:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc6867bf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6801a2160 con 0x7fc6801058e0 2023-08-11T15:45:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc6867bf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6801a2320 con 0x7fc680068440 2023-08-11T15:45:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc680068440 0x7fc68019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:23.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.336+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc680068440 0x7fc68019d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34438/0 (socket says 172.21.15.32:34438) 2023-08-11T15:45:23.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67ffff700 1 -- 172.21.15.32:0/451633604 learned_addr learned my addr 172.21.15.32:0/451633604 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:23.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67f7fe700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6801038b0 0x7fc68019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:23.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67ffff700 1 -- 172.21.15.32:0/451633604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6801058e0 msgr2=0x7fc6801a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:23.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67ffff700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6801058e0 0x7fc6801a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67ffff700 1 -- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6801038b0 msgr2=0x7fc68019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67ffff700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6801038b0 0x7fc68019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67ffff700 1 -- 172.21.15.32:0/451633604 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6700095f0 con 0x7fc680068440 2023-08-11T15:45:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.337+0000 7fc67ffff700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc680068440 0x7fc68019d030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc670005910 tx=0x7fc6700057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.338+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67001d070 con 0x7fc680068440 2023-08-11T15:45:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.338+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc670022470 con 0x7fc680068440 2023-08-11T15:45:23.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.338+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670017650 con 0x7fc680068440 2023-08-11T15:45:23.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.338+0000 7fc6867bf700 1 -- 172.21.15.32:0/451633604 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6801a25a0 con 0x7fc680068440 2023-08-11T15:45:23.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.338+0000 7fc6867bf700 1 -- 172.21.15.32:0/451633604 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6801a9c90 con 0x7fc680068440 2023-08-11T15:45:23.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.340+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc660000fc0 con 0x7fc680068440 2023-08-11T15:45:23.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.345+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc6700225e0 con 0x7fc680068440 2023-08-11T15:45:23.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.346+0000 7fc67d7fa700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc6680a00e0 0x7fc6680a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:23.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.346+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc6700ccaf0 con 0x7fc680068440 2023-08-11T15:45:23.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.347+0000 7fc67d7fa700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] conn(0x7fc6680ab100 0x7fc6680ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.347+0000 7fc67f7fe700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc6680a00e0 0x7fc6680a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.347+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 --> [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6680adc70 con 0x7fc6680ab100 2023-08-11T15:45:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.347+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6700ccea0 con 0x7fc680068440 2023-08-11T15:45:23.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.347+0000 7fc684d5c700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] conn(0x7fc6680ab100 0x7fc6680ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:23.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.348+0000 7fc67f7fe700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc6680a00e0 0x7fc6680a25a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc68019e650 tx=0x7fc674009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:23.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.348+0000 7fc684d5c700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] conn(0x7fc6680ab100 0x7fc6680ad5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:23.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.349+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== osd.7 v2:172.21.15.134:6816/4228806294 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc6680adc70 con 0x7fc6680ab100 2023-08-11T15:45:23.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.372+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 --> [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc660002d40 con 0x7fc6680ab100 2023-08-11T15:45:23.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc67d7fa700 1 -- 172.21.15.32:0/451633604 <== osd.7 v2:172.21.15.134:6816/4228806294 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc660002d40 con 0x7fc6680ab100 2023-08-11T15:45:23.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 >> [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] conn(0x7fc6680ab100 msgr2=0x7fc6680ad5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:23.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc666ffd700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] conn(0x7fc6680ab100 0x7fc6680ad5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc6680a00e0 msgr2=0x7fc6680a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc666ffd700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc6680a00e0 0x7fc6680a25a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc68019e650 tx=0x7fc674009260 comp rx=0 tx=0).stop 2023-08-11T15:45:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc680068440 msgr2=0x7fc68019d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc666ffd700 1 --2- 172.21.15.32:0/451633604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc680068440 0x7fc68019d030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc670005910 tx=0x7fc6700057d0 comp rx=0 tx=0).stop 2023-08-11T15:45:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.373+0000 7fc684d5c700 1 -- 172.21.15.32:0/451633604 reap_dead start 2023-08-11T15:45:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.374+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 shutdown_connections 2023-08-11T15:45:23.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.374+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 >> 172.21.15.32:0/451633604 conn(0x7fc680076920 msgr2=0x7fc6800fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:23.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.374+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 shutdown_connections 2023-08-11T15:45:23.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.374+0000 7fc666ffd700 1 -- 172.21.15.32:0/451633604 wait complete. 2023-08-11T15:45:23.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:23.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:23 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 -- 172.21.15.32:0/794464813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc072cf0 msgr2=0x7f14dc0730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 --2- 172.21.15.32:0/794464813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc072cf0 0x7f14dc0730d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009990 tx=0x7f14cc009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 -- 172.21.15.32:0/794464813 shutdown_connections 2023-08-11T15:45:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 --2- 172.21.15.32:0/794464813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14dc10d020 0x7f14dc10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 --2- 172.21.15.32:0/794464813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc072cf0 0x7f14dc0730d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 --2- 172.21.15.32:0/794464813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc073610 0x7f14dc10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 -- 172.21.15.32:0/794464813 >> 172.21.15.32:0/794464813 conn(0x7f14dc0fc2d0 msgr2=0x7f14dc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 -- 172.21.15.32:0/794464813 shutdown_connections 2023-08-11T15:45:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.861+0000 7f14dbfff700 1 -- 172.21.15.32:0/794464813 wait complete. 2023-08-11T15:45:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.863+0000 7f14dbfff700 1 Processor -- start 2023-08-11T15:45:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.863+0000 7f14dbfff700 1 -- start start 2023-08-11T15:45:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14dc072cf0 0x7f14dc10c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14dbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc073610 0x7f14dc10cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14dbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc10d020 0x7f14dc109790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14dbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc109f60 con 0x7f14dc072cf0 2023-08-11T15:45:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14dbfff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc10a0d0 con 0x7f14dc073610 2023-08-11T15:45:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14dbfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc10a290 con 0x7f14dc10d020 2023-08-11T15:45:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14da7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc073610 0x7f14dc10cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14da7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc073610 0x7f14dc10cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:60438/0 (socket says 172.21.15.32:60438) 2023-08-11T15:45:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14da7fc700 1 -- 172.21.15.32:0/3051589980 learned_addr learned my addr 172.21.15.32:0/3051589980 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14db7fe700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc10d020 0x7f14dc109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14da7fc700 1 -- 172.21.15.32:0/3051589980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc10d020 msgr2=0x7f14dc109790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14da7fc700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc10d020 0x7f14dc109790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14da7fc700 1 -- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14dc072cf0 msgr2=0x7f14dc10c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.864+0000 7f14daffd700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14dc072cf0 0x7f14dc10c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14da7fc700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14dc072cf0 0x7f14dc10c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14da7fc700 1 -- 172.21.15.32:0/3051589980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0095f0 con 0x7f14dc073610 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14daffd700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14dc072cf0 0x7f14dc10c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:45:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14da7fc700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc073610 0x7f14dc10cab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d950 tx=0x7f14d000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0009940 con 0x7f14dc073610 2023-08-11T15:45:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14d0010460 con 0x7f14dc073610 2023-08-11T15:45:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d000f620 con 0x7f14dc073610 2023-08-11T15:45:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc1a9810 con 0x7f14dc073610 2023-08-11T15:45:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.865+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc1a9c30 con 0x7f14dc073610 2023-08-11T15:45:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.866+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14dc04ee70 con 0x7f14dc073610 2023-08-11T15:45:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.869+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f14d00108d0 con 0x7f14dc073610 2023-08-11T15:45:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.870+0000 7f14e0a24700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f14c40a0060 0x7f14c40a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.870+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f14d00cb840 con 0x7f14dc073610 2023-08-11T15:45:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.870+0000 7f14daffd700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f14c40a0060 0x7f14c40a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.870+0000 7f14e0a24700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] conn(0x7f14c40ab080 0x7f14c40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.870+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 --> [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14c40adbf0 con 0x7f14c40ab080 2023-08-11T15:45:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.870+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d00cbbb0 con 0x7f14dc073610 2023-08-11T15:45:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.871+0000 7f14db7fe700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] conn(0x7f14c40ab080 0x7f14c40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.871+0000 7f14daffd700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f14c40a0060 0x7f14c40a2520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00b5c0 tx=0x7f14cc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.871+0000 7f14db7fe700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] conn(0x7f14c40ab080 0x7f14c40ad520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.872+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== osd.9 v2:172.21.15.32:6826/1556490683 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f14c40adbf0 con 0x7f14c40ab080 2023-08-11T15:45:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.890+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 --> [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f14dc112c50 con 0x7f14c40ab080 2023-08-11T15:45:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.891+0000 7f14e0a24700 1 -- 172.21.15.32:0/3051589980 <== osd.9 v2:172.21.15.32:6826/1556490683 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f14d400f040 con 0x7f14c40ab080 2023-08-11T15:45:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.891+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] conn(0x7f14c40ab080 msgr2=0x7f14c40ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.891+0000 7f14dbfff700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] conn(0x7f14c40ab080 0x7f14c40ad520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f14c40a0060 msgr2=0x7f14c40a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f14c40a0060 0x7f14c40a2520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00b5c0 tx=0x7f14cc005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc073610 msgr2=0x7f14dc10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 --2- 172.21.15.32:0/3051589980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14dc073610 0x7f14dc10cab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d950 tx=0x7f14d000dc60 comp rx=0 tx=0).stop 2023-08-11T15:45:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14db7fe700 1 -- 172.21.15.32:0/3051589980 reap_dead start 2023-08-11T15:45:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 shutdown_connections 2023-08-11T15:45:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 >> 172.21.15.32:0/3051589980 conn(0x7f14dc0fc2d0 msgr2=0x7f14dc10f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 shutdown_connections 2023-08-11T15:45:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.892+0000 7f14dbfff700 1 -- 172.21.15.32:0/3051589980 wait complete. 2023-08-11T15:45:24.140 INFO:teuthology.orchestra.run.smithi032.stdout:111669149756 2023-08-11T15:45:24.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-08-11T15:45:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.125+0000 7f29b6501700 1 -- 172.21.15.32:0/1666981290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29b0106940 msgr2=0x7f29b0106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.125+0000 7f29b6501700 1 --2- 172.21.15.32:0/1666981290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29b0106940 0x7f29b0106d20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098f0 tx=0x7f29a0009c00 comp rx=0 tx=0).stop 2023-08-11T15:45:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.126+0000 7f29b6501700 1 -- 172.21.15.32:0/1666981290 shutdown_connections 2023-08-11T15:45:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.126+0000 7f29b6501700 1 --2- 172.21.15.32:0/1666981290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0101300 0x7f29b010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.126+0000 7f29b6501700 1 --2- 172.21.15.32:0/1666981290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29b0106940 0x7f29b0106d20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.126+0000 7f29b6501700 1 --2- 172.21.15.32:0/1666981290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b0100940 0x7f29b0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.126+0000 7f29b6501700 1 -- 172.21.15.32:0/1666981290 >> 172.21.15.32:0/1666981290 conn(0x7f29b00fc410 msgr2=0x7f29b00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.126+0000 7f29b6501700 1 -- 172.21.15.32:0/1666981290 shutdown_connections 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.126+0000 7f29b6501700 1 -- 172.21.15.32:0/1666981290 wait complete. 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 Processor -- start 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 -- start start 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b0100940 0x7f29b0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0101300 0x7f29b0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29b0106940 0x7f29b00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01a4f80 con 0x7f29b0100940 2023-08-11T15:45:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01a50f0 con 0x7f29b0101300 2023-08-11T15:45:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.127+0000 7f29b6501700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01a52b0 con 0x7f29b0106940 2023-08-11T15:45:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b0100940 0x7f29b0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b0100940 0x7f29b0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42932/0 (socket says 172.21.15.32:42932) 2023-08-11T15:45:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29affff700 1 -- 172.21.15.32:0/1502305580 learned_addr learned my addr 172.21.15.32:0/1502305580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29af7fe700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0101300 0x7f29b0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29b4a9e700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29b0106940 0x7f29b00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29af7fe700 1 -- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29b0106940 msgr2=0x7f29b00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29af7fe700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29b0106940 0x7f29b00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29af7fe700 1 -- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b0100940 msgr2=0x7f29b0072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29af7fe700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b0100940 0x7f29b0072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29af7fe700 1 -- 172.21.15.32:0/1502305580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009520 con 0x7f29b0101300 2023-08-11T15:45:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.128+0000 7f29affff700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b0100940 0x7f29b0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.129+0000 7f29af7fe700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0101300 0x7f29b0073160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29a400e9c0 tx=0x7f29a400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.129+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a400c9c0 con 0x7f29b0101300 2023-08-11T15:45:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.129+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a4010430 con 0x7f29b0101300 2023-08-11T15:45:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.129+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b01a5590 con 0x7f29b0101300 2023-08-11T15:45:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.129+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4018680 con 0x7f29b0101300 2023-08-11T15:45:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.129+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b01a5a90 con 0x7f29b0101300 2023-08-11T15:45:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.130+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29b0110be0 con 0x7f29b0101300 2023-08-11T15:45:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.132+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f29a4010860 con 0x7f29b0101300 2023-08-11T15:45:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.134+0000 7f29ad7fa700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f299809ffe0 0x7f29980a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.134+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f29a4014070 con 0x7f29b0101300 2023-08-11T15:45:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.134+0000 7f29affff700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f299809ffe0 0x7f29980a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.134+0000 7f29ad7fa700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] conn(0x7f29980ab000 0x7f29980ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.134+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 --> [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29980adb70 con 0x7f29980ab000 2023-08-11T15:45:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.134+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a40cbde0 con 0x7f29b0101300 2023-08-11T15:45:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.135+0000 7f29b4a9e700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] conn(0x7f29980ab000 0x7f29980ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.135+0000 7f29affff700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f299809ffe0 0x7f29980a24a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009fd0 tx=0x7f29a0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.135+0000 7f29b4a9e700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] conn(0x7f29980ab000 0x7f29980ad4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.136+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== osd.11 v2:172.21.15.192:6824/1267688108 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f29980adb70 con 0x7f29980ab000 2023-08-11T15:45:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.154+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 --> [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29b004e620 con 0x7f29980ab000 2023-08-11T15:45:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.154+0000 7f29ad7fa700 1 -- 172.21.15.32:0/1502305580 <== osd.11 v2:172.21.15.192:6824/1267688108 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f29b004e620 con 0x7f29980ab000 2023-08-11T15:45:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] conn(0x7f29980ab000 msgr2=0x7f29980ad4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] conn(0x7f29980ab000 0x7f29980ad4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f299809ffe0 msgr2=0x7f29980a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f299809ffe0 0x7f29980a24a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009fd0 tx=0x7f29a0019040 comp rx=0 tx=0).stop 2023-08-11T15:45:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0101300 msgr2=0x7f29b0073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 --2- 172.21.15.32:0/1502305580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0101300 0x7f29b0073160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29a400e9c0 tx=0x7f29a400ecd0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b4a9e700 1 -- 172.21.15.32:0/1502305580 reap_dead start 2023-08-11T15:45:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 shutdown_connections 2023-08-11T15:45:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 >> 172.21.15.32:0/1502305580 conn(0x7f29b00fc410 msgr2=0x7f29b00fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 shutdown_connections 2023-08-11T15:45:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.155+0000 7f29b6501700 1 -- 172.21.15.32:0/1502305580 wait complete. 2023-08-11T15:45:24.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:23 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:24.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:23 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:24.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:23 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:24.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:23 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 -- 172.21.15.32:0/1621638639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ccc100940 msgr2=0x7f1ccc100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 --2- 172.21.15.32:0/1621638639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ccc100940 0x7f1ccc100dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009990 tx=0x7f1cc0009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 -- 172.21.15.32:0/1621638639 shutdown_connections 2023-08-11T15:45:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 --2- 172.21.15.32:0/1621638639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ccc106940 0x7f1ccc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 --2- 172.21.15.32:0/1621638639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ccc101300 0x7f1ccc10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 --2- 172.21.15.32:0/1621638639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ccc100940 0x7f1ccc100dc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 -- 172.21.15.32:0/1621638639 >> 172.21.15.32:0/1621638639 conn(0x7f1ccc0fc410 msgr2=0x7f1ccc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 -- 172.21.15.32:0/1621638639 shutdown_connections 2023-08-11T15:45:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.934+0000 7f1cd2121700 1 -- 172.21.15.32:0/1621638639 wait complete. 2023-08-11T15:45:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.935+0000 7f1cd2121700 1 Processor -- start 2023-08-11T15:45:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1cd2121700 1 -- start start 2023-08-11T15:45:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1cd2121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ccc100940 0x7f1ccc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1cd2121700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ccc101300 0x7f1ccc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1cd2121700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ccc106940 0x7f1ccc196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1cd2121700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc196eb0 con 0x7f1ccc100940 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1cd2121700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc197020 con 0x7f1ccc101300 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1cd2121700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc1971e0 con 0x7f1ccc106940 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ccc100940 0x7f1ccc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ccc100940 0x7f1ccc19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42918/0 (socket says 172.21.15.32:42918) 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1ccb7fe700 1 -- 172.21.15.32:0/2182429487 learned_addr learned my addr 172.21.15.32:0/2182429487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1ccbfff700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ccc106940 0x7f1ccc196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.936+0000 7f1ccaffd700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ccc101300 0x7f1ccc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1ccb7fe700 1 -- 172.21.15.32:0/2182429487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ccc106940 msgr2=0x7f1ccc196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1ccb7fe700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ccc106940 0x7f1ccc196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1ccb7fe700 1 -- 172.21.15.32:0/2182429487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ccc101300 msgr2=0x7f1ccc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1ccb7fe700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ccc101300 0x7f1ccc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1ccb7fe700 1 -- 172.21.15.32:0/2182429487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00095f0 con 0x7f1ccc100940 2023-08-11T15:45:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1ccb7fe700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ccc100940 0x7f1ccc19c7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e9a0 tx=0x7f1cbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc009790 con 0x7f1ccc100940 2023-08-11T15:45:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc1974c0 con 0x7f1ccc100940 2023-08-11T15:45:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cbc010430 con 0x7f1ccc100940 2023-08-11T15:45:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc018680 con 0x7f1ccc100940 2023-08-11T15:45:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.937+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc1a37b0 con 0x7f1ccc100940 2023-08-11T15:45:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.938+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ccc04e620 con 0x7f1ccc100940 2023-08-11T15:45:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.940+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f1cbc0105a0 con 0x7f1ccc100940 2023-08-11T15:45:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.942+0000 7f1cc8ff9700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1cb409ffe0 0x7f1cb40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.942+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1cbc014070 con 0x7f1ccc100940 2023-08-11T15:45:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.942+0000 7f1ccaffd700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1cb409ffe0 0x7f1cb40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.942+0000 7f1cc8ff9700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] conn(0x7f1cb40ab000 0x7f1cb40ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.942+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 --> [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cb40adb70 con 0x7f1cb40ab000 2023-08-11T15:45:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.942+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cbc0cbe90 con 0x7f1ccc100940 2023-08-11T15:45:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.942+0000 7f1ccbfff700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] conn(0x7f1cb40ab000 0x7f1cb40ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.943+0000 7f1ccaffd700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1cb409ffe0 0x7f1cb40a24a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000b2e0 tx=0x7f1cc0005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.943+0000 7f1ccbfff700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] conn(0x7f1cb40ab000 0x7f1cb40ad4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.944+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== osd.6 v2:172.21.15.32:6818/846261064 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1cb40adb70 con 0x7f1cb40ab000 2023-08-11T15:45:24.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.967+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 --> [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ccc110c30 con 0x7f1cb40ab000 2023-08-11T15:45:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.968+0000 7f1cc8ff9700 1 -- 172.21.15.32:0/2182429487 <== osd.6 v2:172.21.15.32:6818/846261064 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1cc400f040 con 0x7f1cb40ab000 2023-08-11T15:45:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.968+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] conn(0x7f1cb40ab000 msgr2=0x7f1cb40ad4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.968+0000 7f1cd2121700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] conn(0x7f1cb40ab000 0x7f1cb40ad4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1cb409ffe0 msgr2=0x7f1cb40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f1cb409ffe0 0x7f1cb40a24a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000b2e0 tx=0x7f1cc0005960 comp rx=0 tx=0).stop 2023-08-11T15:45:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ccc100940 msgr2=0x7f1ccc19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 --2- 172.21.15.32:0/2182429487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ccc100940 0x7f1ccc19c7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e9a0 tx=0x7f1cbc00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1ccbfff700 1 -- 172.21.15.32:0/2182429487 reap_dead start 2023-08-11T15:45:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 shutdown_connections 2023-08-11T15:45:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 >> 172.21.15.32:0/2182429487 conn(0x7f1ccc0fc410 msgr2=0x7f1ccc104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 shutdown_connections 2023-08-11T15:45:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.969+0000 7f1cd2121700 1 -- 172.21.15.32:0/2182429487 wait complete. 2023-08-11T15:45:24.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.536+0000 7f3176c47700 1 -- 172.21.15.32:0/128302105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f317010d190 msgr2=0x7f317010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.536+0000 7f3176c47700 1 --2- 172.21.15.32:0/128302105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f317010d190 0x7f317010f600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f316800b0a0 tx=0x7f316800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.536+0000 7f3176c47700 1 -- 172.21.15.32:0/128302105 shutdown_connections 2023-08-11T15:45:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.537+0000 7f3176c47700 1 --2- 172.21.15.32:0/128302105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f317010d190 0x7f317010f600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.537+0000 7f3176c47700 1 --2- 172.21.15.32:0/128302105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3170072d50 0x7f3170073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.537+0000 7f3176c47700 1 --2- 172.21.15.32:0/128302105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3170073670 0x7f317010cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.537+0000 7f3176c47700 1 -- 172.21.15.32:0/128302105 >> 172.21.15.32:0/128302105 conn(0x7f31700fc440 msgr2=0x7f31700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.537+0000 7f3176c47700 1 -- 172.21.15.32:0/128302105 shutdown_connections 2023-08-11T15:45:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.537+0000 7f3176c47700 1 -- 172.21.15.32:0/128302105 wait complete. 2023-08-11T15:45:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 Processor -- start 2023-08-11T15:45:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 -- start start 2023-08-11T15:45:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3170072d50 0x7f3170198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3170073670 0x7f31701992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f317010d190 0x7f317019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019dd20 con 0x7f317010d190 2023-08-11T15:45:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019de90 con 0x7f3170073670 2023-08-11T15:45:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.538+0000 7f3176c47700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019e050 con 0x7f3170072d50 2023-08-11T15:45:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3170072d50 0x7f3170198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3170072d50 0x7f3170198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34520/0 (socket says 172.21.15.32:34520) 2023-08-11T15:45:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 -- 172.21.15.32:0/3454870355 learned_addr learned my addr 172.21.15.32:0/3454870355 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31751e4700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f317010d190 0x7f317019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f316ffff700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3170073670 0x7f31701992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 -- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3170073670 msgr2=0x7f31701992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3170073670 0x7f31701992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 -- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f317010d190 msgr2=0x7f317019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f317010d190 0x7f317019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 -- 172.21.15.32:0/3454870355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3168009d00 con 0x7f3170072d50 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.539+0000 7f31749e3700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3170072d50 0x7f3170198d60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f316400d980 tx=0x7f316400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.540+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164004220 con 0x7f3170072d50 2023-08-11T15:45:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.540+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317019e330 con 0x7f3170072d50 2023-08-11T15:45:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.540+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3164007640 con 0x7f3170072d50 2023-08-11T15:45:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.540+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640055c0 con 0x7f3170072d50 2023-08-11T15:45:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.540+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31701a5b90 con 0x7f3170072d50 2023-08-11T15:45:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.541+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3170112d30 con 0x7f3170072d50 2023-08-11T15:45:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.544+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f3164004380 con 0x7f3170072d50 2023-08-11T15:45:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.546+0000 7f316dffb700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31600a0060 0x7f31600a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.547+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f31640cbb10 con 0x7f3170072d50 2023-08-11T15:45:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.547+0000 7f316ffff700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31600a0060 0x7f31600a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.547+0000 7f316dffb700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] conn(0x7f31600ab080 0x7f31600ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.547+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 --> [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31600adbf0 con 0x7f31600ab080 2023-08-11T15:45:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.547+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31640cbe80 con 0x7f3170072d50 2023-08-11T15:45:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.548+0000 7f31751e4700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] conn(0x7f31600ab080 0x7f31600ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.548+0000 7f316ffff700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31600a0060 0x7f31600a2520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f317019a380 tx=0x7f315c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.548+0000 7f31751e4700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] conn(0x7f31600ab080 0x7f31600ad520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.549+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== osd.1 v2:172.21.15.134:6800/251950255 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f31600adbf0 con 0x7f31600ab080 2023-08-11T15:45:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.569+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 --> [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f317004e620 con 0x7f31600ab080 2023-08-11T15:45:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f316dffb700 1 -- 172.21.15.32:0/3454870355 <== osd.1 v2:172.21.15.134:6800/251950255 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f317004e620 con 0x7f31600ab080 2023-08-11T15:45:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] conn(0x7f31600ab080 msgr2=0x7f31600ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f3176c47700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] conn(0x7f31600ab080 0x7f31600ad520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31600a0060 msgr2=0x7f31600a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f3176c47700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31600a0060 0x7f31600a2520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f317019a380 tx=0x7f315c009260 comp rx=0 tx=0).stop 2023-08-11T15:45:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3170072d50 msgr2=0x7f3170198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f3176c47700 1 --2- 172.21.15.32:0/3454870355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3170072d50 0x7f3170198d60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f316400d980 tx=0x7f316400dc90 comp rx=0 tx=0).stop 2023-08-11T15:45:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.570+0000 7f31751e4700 1 -- 172.21.15.32:0/3454870355 reap_dead start 2023-08-11T15:45:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.571+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 shutdown_connections 2023-08-11T15:45:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.571+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 >> 172.21.15.32:0/3454870355 conn(0x7f31700fc440 msgr2=0x7f3170104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.571+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 shutdown_connections 2023-08-11T15:45:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:24.571+0000 7f3176c47700 1 -- 172.21.15.32:0/3454870355 wait complete. 2023-08-11T15:45:24.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.521+0000 7fa50fb86700 1 -- 172.21.15.32:0/3332613775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 msgr2=0x7fa508073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.521+0000 7fa50fb86700 1 --2- 172.21.15.32:0/3332613775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 0x7fa508073130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa504009990 tx=0x7fa504009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.521+0000 7fa50fb86700 1 -- 172.21.15.32:0/3332613775 shutdown_connections 2023-08-11T15:45:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.521+0000 7fa50fb86700 1 --2- 172.21.15.32:0/3332613775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa50810d180 0x7fa50810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.521+0000 7fa50fb86700 1 --2- 172.21.15.32:0/3332613775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 0x7fa508073130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.521+0000 7fa50fb86700 1 --2- 172.21.15.32:0/3332613775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa508073670 0x7fa50810cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.521+0000 7fa50fb86700 1 -- 172.21.15.32:0/3332613775 >> 172.21.15.32:0/3332613775 conn(0x7fa5080fc410 msgr2=0x7fa5080fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.522+0000 7fa50fb86700 1 -- 172.21.15.32:0/3332613775 shutdown_connections 2023-08-11T15:45:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.523+0000 7fa50fb86700 1 -- 172.21.15.32:0/3332613775 wait complete. 2023-08-11T15:45:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.524+0000 7fa50fb86700 1 Processor -- start 2023-08-11T15:45:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.525+0000 7fa50fb86700 1 -- start start 2023-08-11T15:45:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.525+0000 7fa50fb86700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 0x7fa508198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.525+0000 7fa50fb86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa508073670 0x7fa5081991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.525+0000 7fa50fb86700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa50810d180 0x7fa50819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.525+0000 7fa50fb86700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50819cba0 con 0x7fa508073670 2023-08-11T15:45:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.525+0000 7fa50fb86700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50819cd10 con 0x7fa50810d180 2023-08-11T15:45:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.525+0000 7fa50fb86700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50819ced0 con 0x7fa508072d50 2023-08-11T15:45:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.526+0000 7fa50d121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa508073670 0x7fa5081991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.526+0000 7fa50d922700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 0x7fa508198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.526+0000 7fa50d121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa508073670 0x7fa5081991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42874/0 (socket says 172.21.15.32:42874) 2023-08-11T15:45:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.526+0000 7fa50d121700 1 -- 172.21.15.32:0/2406890689 learned_addr learned my addr 172.21.15.32:0/2406890689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.526+0000 7fa50e123700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa50810d180 0x7fa50819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.527+0000 7fa50d922700 1 -- 172.21.15.32:0/2406890689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa50810d180 msgr2=0x7fa50819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.527+0000 7fa50d922700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa50810d180 0x7fa50819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.527+0000 7fa50d922700 1 -- 172.21.15.32:0/2406890689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa508073670 msgr2=0x7fa5081991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.527+0000 7fa50d922700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa508073670 0x7fa5081991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.527+0000 7fa50d922700 1 -- 172.21.15.32:0/2406890689 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040095f0 con 0x7fa508072d50 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.527+0000 7fa50d922700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 0x7fa508198ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa504009990 tx=0x7fa504005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.528+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50401d070 con 0x7fa508072d50 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.528+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50819d150 con 0x7fa508072d50 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.528+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa504022470 con 0x7fa508072d50 2023-08-11T15:45:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.528+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504017650 con 0x7fa508072d50 2023-08-11T15:45:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.528+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5081a5b00 con 0x7fa508072d50 2023-08-11T15:45:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.529+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa508112db0 con 0x7fa508072d50 2023-08-11T15:45:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.531+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa504022870 con 0x7fa508072d50 2023-08-11T15:45:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.534+0000 7fa4feffd700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa4f40a0060 0x7fa4f40a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.534+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa5040ccb80 con 0x7fa508072d50 2023-08-11T15:45:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.534+0000 7fa50d121700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa4f40a0060 0x7fa4f40a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.535+0000 7fa4feffd700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] conn(0x7fa4f40ab080 0x7fa4f40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:24.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.535+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 --> [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4f40adbf0 con 0x7fa4f40ab080 2023-08-11T15:45:24.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.535+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5040ccef0 con 0x7fa508072d50 2023-08-11T15:45:24.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.535+0000 7fa50e123700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] conn(0x7fa4f40ab080 0x7fa4f40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:24.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.535+0000 7fa50d121700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa4f40a0060 0x7fa4f40a2520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa50819a280 tx=0x7fa4f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.536+0000 7fa50e123700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] conn(0x7fa4f40ab080 0x7fa4f40ad520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:24.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.537+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== osd.5 v2:172.21.15.192:6808/3471337783 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa4f40adbf0 con 0x7fa4f40ab080 2023-08-11T15:45:24.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.566+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 --> [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa50804e620 con 0x7fa4f40ab080 2023-08-11T15:45:24.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa4feffd700 1 -- 172.21.15.32:0/2406890689 <== osd.5 v2:172.21.15.192:6808/3471337783 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa50804e620 con 0x7fa4f40ab080 2023-08-11T15:45:24.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] conn(0x7fa4f40ab080 msgr2=0x7fa4f40ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa50fb86700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] conn(0x7fa4f40ab080 0x7fa4f40ad520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:24.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa4f40a0060 msgr2=0x7fa4f40a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa50fb86700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa4f40a0060 0x7fa4f40a2520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa50819a280 tx=0x7fa4f8009260 comp rx=0 tx=0).stop 2023-08-11T15:45:24.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 msgr2=0x7fa508198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:24.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa50fb86700 1 --2- 172.21.15.32:0/2406890689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa508072d50 0x7fa508198ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa504009990 tx=0x7fa504005740 comp rx=0 tx=0).stop 2023-08-11T15:45:24.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.567+0000 7fa50e123700 1 -- 172.21.15.32:0/2406890689 reap_dead start 2023-08-11T15:45:24.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.568+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 shutdown_connections 2023-08-11T15:45:24.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.568+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 >> 172.21.15.32:0/2406890689 conn(0x7fa5080fc410 msgr2=0x7fa50810f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:24.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.568+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 shutdown_connections 2023-08-11T15:45:24.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:23.568+0000 7fa50fb86700 1 -- 172.21.15.32:0/2406890689 wait complete. 2023-08-11T15:45:24.950 INFO:teuthology.orchestra.run.smithi032.stdout:309237645340 2023-08-11T15:45:24.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-08-11T15:45:25.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:24 smithi032 ceph-mon[97103]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:25.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:24 smithi134 ceph-mon[130447]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.176+0000 7fd208077700 1 -- 172.21.15.32:0/3037353519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2000688e0 msgr2=0x7fd200068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.176+0000 7fd208077700 1 --2- 172.21.15.32:0/3037353519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2000688e0 0x7fd200068cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0099c0 tx=0x7fd1fc009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.177+0000 7fd208077700 1 -- 172.21.15.32:0/3037353519 shutdown_connections 2023-08-11T15:45:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.177+0000 7fd208077700 1 --2- 172.21.15.32:0/3037353519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2000688e0 0x7fd200068cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.177+0000 7fd208077700 1 --2- 172.21.15.32:0/3037353519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd200105d50 0x7fd20010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.177+0000 7fd208077700 1 --2- 172.21.15.32:0/3037353519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd200105390 0x7fd200105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.177+0000 7fd208077700 1 -- 172.21.15.32:0/3037353519 >> 172.21.15.32:0/3037353519 conn(0x7fd200076920 msgr2=0x7fd200076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.177+0000 7fd208077700 1 -- 172.21.15.32:0/3037353519 shutdown_connections 2023-08-11T15:45:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.177+0000 7fd208077700 1 -- 172.21.15.32:0/3037353519 wait complete. 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.178+0000 7fd208077700 1 Processor -- start 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.178+0000 7fd208077700 1 -- start start 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd208077700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2000688e0 0x7fd20019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd208077700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd200105390 0x7fd20019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd208077700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd200105d50 0x7fd200198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd208077700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2001990b0 con 0x7fd200105390 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd208077700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd200199220 con 0x7fd200105d50 2023-08-11T15:45:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd208077700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2001993e0 con 0x7fd2000688e0 2023-08-11T15:45:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd205e13700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2000688e0 0x7fd20019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd206614700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd200105d50 0x7fd200198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd205e13700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2000688e0 0x7fd20019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:48616/0 (socket says 172.21.15.32:48616) 2023-08-11T15:45:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd205e13700 1 -- 172.21.15.32:0/2942593925 learned_addr learned my addr 172.21.15.32:0/2942593925 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.179+0000 7fd205612700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd200105390 0x7fd20019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd206614700 1 -- 172.21.15.32:0/2942593925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2000688e0 msgr2=0x7fd20019e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd206614700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2000688e0 0x7fd20019e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd206614700 1 -- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd200105390 msgr2=0x7fd20019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd206614700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd200105390 0x7fd20019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd206614700 1 -- 172.21.15.32:0/2942593925 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0095f0 con 0x7fd200105d50 2023-08-11T15:45:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd206614700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd200105d50 0x7fd200198a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007f10 tx=0x7fd1f800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8004db0 con 0x7fd200105d50 2023-08-11T15:45:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1f800f040 con 0x7fd200105d50 2023-08-11T15:45:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2001996c0 con 0x7fd200105d50 2023-08-11T15:45:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.180+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8005830 con 0x7fd200105d50 2023-08-11T15:45:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.181+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2001a59b0 con 0x7fd200105d50 2023-08-11T15:45:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.181+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd20004e620 con 0x7fd200105d50 2023-08-11T15:45:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.184+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd1f800db20 con 0x7fd200105d50 2023-08-11T15:45:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.186+0000 7fd1f6ffd700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1ec0a0060 0x7fd1ec0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.186+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1f80cb9f0 con 0x7fd200105d50 2023-08-11T15:45:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.186+0000 7fd205e13700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1ec0a0060 0x7fd1ec0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.187+0000 7fd1f6ffd700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] conn(0x7fd1ec0ab080 0x7fd1ec0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.187+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 --> [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1ec0adbf0 con 0x7fd1ec0ab080 2023-08-11T15:45:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.187+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f80cbd60 con 0x7fd200105d50 2023-08-11T15:45:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.187+0000 7fd205e13700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1ec0a0060 0x7fd1ec0a2520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc003760 tx=0x7fd1fc0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.187+0000 7fd205612700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] conn(0x7fd1ec0ab080 0x7fd1ec0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.188+0000 7fd205612700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] conn(0x7fd1ec0ab080 0x7fd1ec0ad520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.190+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== osd.2 v2:172.21.15.192:6800/2619008811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd1ec0adbf0 con 0x7fd1ec0ab080 2023-08-11T15:45:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.208+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 --> [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd20010c3e0 con 0x7fd1ec0ab080 2023-08-11T15:45:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.209+0000 7fd1f6ffd700 1 -- 172.21.15.32:0/2942593925 <== osd.2 v2:172.21.15.192:6800/2619008811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd20010c3e0 con 0x7fd1ec0ab080 2023-08-11T15:45:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.210+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 >> [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] conn(0x7fd1ec0ab080 msgr2=0x7fd1ec0ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.210+0000 7fd208077700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] conn(0x7fd1ec0ab080 0x7fd1ec0ad520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.210+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1ec0a0060 msgr2=0x7fd1ec0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.210+0000 7fd208077700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd1ec0a0060 0x7fd1ec0a2520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc003760 tx=0x7fd1fc0058e0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.210+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd200105d50 msgr2=0x7fd200198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.210+0000 7fd208077700 1 --2- 172.21.15.32:0/2942593925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd200105d50 0x7fd200198a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007f10 tx=0x7fd1f800d3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.211+0000 7fd206614700 1 -- 172.21.15.32:0/2942593925 reap_dead start 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.211+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 shutdown_connections 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.211+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 >> 172.21.15.32:0/2942593925 conn(0x7fd200076920 msgr2=0x7fd20010ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.211+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 shutdown_connections 2023-08-11T15:45:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:25.211+0000 7fd208077700 1 -- 172.21.15.32:0/2942593925 wait complete. 2023-08-11T15:45:25.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:24 smithi192 ceph-mon[130806]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:45:25.529 INFO:teuthology.orchestra.run.smithi032.stdout:287762808866 2023-08-11T15:45:25.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-08-11T15:45:25.939 INFO:teuthology.orchestra.run.smithi032.stdout:223338299435 2023-08-11T15:45:25.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-08-11T15:45:25.961 INFO:teuthology.orchestra.run.smithi032.stdout:339302416409 2023-08-11T15:45:25.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-08-11T15:45:26.286 INFO:teuthology.orchestra.run.smithi032.stdout:77309411395 2023-08-11T15:45:26.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-08-11T15:45:26.542 INFO:teuthology.orchestra.run.smithi032.stdout:55834574918 2023-08-11T15:45:26.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-08-11T15:45:26.902 INFO:teuthology.orchestra.run.smithi032.stdout:167503724596 2023-08-11T15:45:26.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-08-11T15:45:27.193 INFO:teuthology.orchestra.run.smithi032.stdout:201863462960 2023-08-11T15:45:27.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-08-11T15:45:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.297+0000 7fe11e3f9700 1 -- 172.21.15.32:0/2879519298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118106980 msgr2=0x7fe118106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.297+0000 7fe11e3f9700 1 --2- 172.21.15.32:0/2879519298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118106980 0x7fe118106d60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe10c01a9a0 tx=0x7fe10c01acb0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.297+0000 7fe11e3f9700 1 -- 172.21.15.32:0/2879519298 shutdown_connections 2023-08-11T15:45:27.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.297+0000 7fe11e3f9700 1 --2- 172.21.15.32:0/2879519298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118106980 0x7fe118106d60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.297+0000 7fe11e3f9700 1 --2- 172.21.15.32:0/2879519298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe118101340 0x7fe11810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.297+0000 7fe11e3f9700 1 --2- 172.21.15.32:0/2879519298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe118100980 0x7fe118100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.297+0000 7fe11e3f9700 1 -- 172.21.15.32:0/2879519298 >> 172.21.15.32:0/2879519298 conn(0x7fe1180fc450 msgr2=0x7fe1180fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:27.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.300+0000 7fe11e3f9700 1 -- 172.21.15.32:0/2879519298 shutdown_connections 2023-08-11T15:45:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.301+0000 7fe11e3f9700 1 -- 172.21.15.32:0/2879519298 wait complete. 2023-08-11T15:45:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 Processor -- start 2023-08-11T15:45:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 -- start start 2023-08-11T15:45:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118100980 0x7fe1181090f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe118101340 0x7fe118109630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe118106980 0x7fe1181bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181bc670 con 0x7fe118100980 2023-08-11T15:45:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181bc7b0 con 0x7fe118106980 2023-08-11T15:45:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.302+0000 7fe11e3f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181bc970 con 0x7fe118101340 2023-08-11T15:45:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.303+0000 7fe1177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe118101340 0x7fe118109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.303+0000 7fe1177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe118101340 0x7fe118109630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:48634/0 (socket says 172.21.15.32:48634) 2023-08-11T15:45:27.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.303+0000 7fe1177fe700 1 -- 172.21.15.32:0/3431734852 learned_addr learned my addr 172.21.15.32:0/3431734852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:27.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.304+0000 7fe117fff700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118100980 0x7fe1181090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.304+0000 7fe11c996700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe118106980 0x7fe1181bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.305+0000 7fe117fff700 1 -- 172.21.15.32:0/3431734852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe118101340 msgr2=0x7fe118109630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.305+0000 7fe117fff700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe118101340 0x7fe118109630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.305+0000 7fe117fff700 1 -- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe118106980 msgr2=0x7fe1181bc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.305+0000 7fe117fff700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe118106980 0x7fe1181bc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.305+0000 7fe117fff700 1 -- 172.21.15.32:0/3431734852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c01a600 con 0x7fe118100980 2023-08-11T15:45:27.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.306+0000 7fe117fff700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118100980 0x7fe1181090f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.306+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003fc0 con 0x7fe118100980 2023-08-11T15:45:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.306+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10c004120 con 0x7fe118100980 2023-08-11T15:45:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.306+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c01f650 con 0x7fe118100980 2023-08-11T15:45:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.307+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1181bcbf0 con 0x7fe118100980 2023-08-11T15:45:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.307+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1181bcfb0 con 0x7fe118100980 2023-08-11T15:45:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.308+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe11806cd90 con 0x7fe118100980 2023-08-11T15:45:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.313+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fe10c00f480 con 0x7fe118100980 2023-08-11T15:45:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.315+0000 7fe1157fa700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1000a00a0 0x7fe1000a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.315+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe10c0d53a0 con 0x7fe118100980 2023-08-11T15:45:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.316+0000 7fe1157fa700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] conn(0x7fe1000ab0c0 0x7fe1000ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.316+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 --> [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1000adc30 con 0x7fe1000ab0c0 2023-08-11T15:45:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.316+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe10c0d56e0 con 0x7fe118100980 2023-08-11T15:45:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.316+0000 7fe1177fe700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1000a00a0 0x7fe1000a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.316+0000 7fe11c996700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] conn(0x7fe1000ab0c0 0x7fe1000ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.317+0000 7fe11c996700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] conn(0x7fe1000ab0c0 0x7fe1000ad560 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.317+0000 7fe1177fe700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1000a00a0 0x7fe1000a2560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe118106060 tx=0x7fe108009210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.318+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== osd.4 v2:172.21.15.134:6808/3721901848 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe1000adc30 con 0x7fe1000ab0c0 2023-08-11T15:45:27.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.348+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 --> [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe11804e620 con 0x7fe1000ab0c0 2023-08-11T15:45:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.348+0000 7fe1157fa700 1 -- 172.21.15.32:0/3431734852 <== osd.4 v2:172.21.15.134:6808/3721901848 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe11804e620 con 0x7fe1000ab0c0 2023-08-11T15:45:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.349+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] conn(0x7fe1000ab0c0 msgr2=0x7fe1000ad560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.349+0000 7fe11e3f9700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] conn(0x7fe1000ab0c0 0x7fe1000ad560 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.349+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1000a00a0 msgr2=0x7fe1000a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.350+0000 7fe11e3f9700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1000a00a0 0x7fe1000a2560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe118106060 tx=0x7fe108009210 comp rx=0 tx=0).stop 2023-08-11T15:45:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.350+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118100980 msgr2=0x7fe1181090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.350+0000 7fe11e3f9700 1 --2- 172.21.15.32:0/3431734852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118100980 0x7fe1181090f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c005750 comp rx=0 tx=0).stop 2023-08-11T15:45:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.350+0000 7fe11c996700 1 -- 172.21.15.32:0/3431734852 reap_dead start 2023-08-11T15:45:27.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.351+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 shutdown_connections 2023-08-11T15:45:27.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.351+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 >> 172.21.15.32:0/3431734852 conn(0x7fe1180fc450 msgr2=0x7fe1180fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.353+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 shutdown_connections 2023-08-11T15:45:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.353+0000 7fe11e3f9700 1 -- 172.21.15.32:0/3431734852 wait complete. 2023-08-11T15:45:27.403 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:27 smithi032 ceph-mon[97103]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:27.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:27 smithi134 ceph-mon[130447]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:27.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:27 smithi192 ceph-mon[130806]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 -- 172.21.15.32:0/1443401440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800101300 msgr2=0x7f380010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 --2- 172.21.15.32:0/1443401440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800101300 0x7f380010d810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f37f800b0a0 tx=0x7f37f800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 -- 172.21.15.32:0/1443401440 shutdown_connections 2023-08-11T15:45:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 --2- 172.21.15.32:0/1443401440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800101300 0x7f380010d810 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 --2- 172.21.15.32:0/1443401440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800106940 0x7f3800106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 --2- 172.21.15.32:0/1443401440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3800100940 0x7f3800100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 -- 172.21.15.32:0/1443401440 >> 172.21.15.32:0/1443401440 conn(0x7f38000fc410 msgr2=0x7f38000fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 -- 172.21.15.32:0/1443401440 shutdown_connections 2023-08-11T15:45:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.706+0000 7f3807511700 1 -- 172.21.15.32:0/1443401440 wait complete. 2023-08-11T15:45:27.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.708+0000 7f3807511700 1 Processor -- start 2023-08-11T15:45:27.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.708+0000 7f3807511700 1 -- start start 2023-08-11T15:45:27.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3807511700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800100940 0x7f380019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3807511700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3800101300 0x7f380019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3807511700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800106940 0x7f3800196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3807511700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800196fd0 con 0x7f3800106940 2023-08-11T15:45:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3807511700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800197140 con 0x7f3800100940 2023-08-11T15:45:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3807511700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800197300 con 0x7f3800101300 2023-08-11T15:45:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f38052ad700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800100940 0x7f380019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f38052ad700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800100940 0x7f380019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:43910/0 (socket says 172.21.15.32:43910) 2023-08-11T15:45:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f38052ad700 1 -- 172.21.15.32:0/799657630 learned_addr learned my addr 172.21.15.32:0/799657630 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3805aae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800106940 0x7f3800196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3804aac700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3800101300 0x7f380019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3805aae700 1 -- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3800101300 msgr2=0x7f380019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3805aae700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3800101300 0x7f380019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3805aae700 1 -- 172.21.15.32:0/799657630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800100940 msgr2=0x7f380019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3805aae700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800100940 0x7f380019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3805aae700 1 -- 172.21.15.32:0/799657630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8009d00 con 0x7f3800106940 2023-08-11T15:45:27.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f3804aac700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3800101300 0x7f380019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:45:27.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.709+0000 7f38052ad700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800100940 0x7f380019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:27.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.710+0000 7f3805aae700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800106940 0x7f3800196960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37f800b0a0 tx=0x7f37f8007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:27.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.710+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800e040 con 0x7f3800106940 2023-08-11T15:45:27.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.710+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3800197580 con 0x7f3800106940 2023-08-11T15:45:27.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.710+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37f800be40 con 0x7f3800106940 2023-08-11T15:45:27.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.710+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80052a0 con 0x7f3800106940 2023-08-11T15:45:27.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.710+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38001a38d0 con 0x7f3800106940 2023-08-11T15:45:27.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.711+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3800110be0 con 0x7f3800106940 2023-08-11T15:45:27.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.713+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f37f801d470 con 0x7f3800106940 2023-08-11T15:45:27.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.715+0000 7f37f67fc700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f37ec0a0060 0x7f37ec0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.715+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f37f80cd280 con 0x7f3800106940 2023-08-11T15:45:27.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.715+0000 7f38052ad700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f37ec0a0060 0x7f37ec0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.715+0000 7f37f67fc700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] conn(0x7f37ec0ab080 0x7f37ec0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:27.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.715+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 --> [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37ec0adbf0 con 0x7f37ec0ab080 2023-08-11T15:45:27.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.715+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37f80cd5f0 con 0x7f3800106940 2023-08-11T15:45:27.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.716+0000 7f38052ad700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f37ec0a0060 0x7f37ec0a2520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005950 tx=0x7f37fc00b500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:27.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.716+0000 7f3804aac700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] conn(0x7f37ec0ab080 0x7f37ec0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:27.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.716+0000 7f3804aac700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] conn(0x7f37ec0ab080 0x7f37ec0ad520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:27.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.717+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== osd.8 v2:172.21.15.192:6816/2583846672 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f37ec0adbf0 con 0x7f37ec0ab080 2023-08-11T15:45:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.734+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 --> [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f38000fd2e0 con 0x7f37ec0ab080 2023-08-11T15:45:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.735+0000 7f37f67fc700 1 -- 172.21.15.32:0/799657630 <== osd.8 v2:172.21.15.192:6816/2583846672 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f38000fd2e0 con 0x7f37ec0ab080 2023-08-11T15:45:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.735+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] conn(0x7f37ec0ab080 msgr2=0x7f37ec0ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.735+0000 7f3807511700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] conn(0x7f37ec0ab080 0x7f37ec0ad520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.735+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f37ec0a0060 msgr2=0x7f37ec0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.735+0000 7f3807511700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f37ec0a0060 0x7f37ec0a2520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005950 tx=0x7f37fc00b500 comp rx=0 tx=0).stop 2023-08-11T15:45:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.735+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800106940 msgr2=0x7f3800196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.735+0000 7f3807511700 1 --2- 172.21.15.32:0/799657630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3800106940 0x7f3800196960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37f800b0a0 tx=0x7f37f8007990 comp rx=0 tx=0).stop 2023-08-11T15:45:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.736+0000 7f3805aae700 1 -- 172.21.15.32:0/799657630 reap_dead start 2023-08-11T15:45:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.736+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 shutdown_connections 2023-08-11T15:45:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.736+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 >> 172.21.15.32:0/799657630 conn(0x7f38000fc410 msgr2=0x7f38000fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.736+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 shutdown_connections 2023-08-11T15:45:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:27.736+0000 7f3807511700 1 -- 172.21.15.32:0/799657630 wait complete. 2023-08-11T15:45:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:27.788 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:27.976 INFO:teuthology.orchestra.run.smithi032.stdout:146028888122 2023-08-11T15:45:27.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-08-11T15:45:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:29.238 INFO:teuthology.orchestra.run.smithi032.stdout:253403070504 2023-08-11T15:45:29.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-08-11T15:45:29.386 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:29.421 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:29 smithi032 ceph-mon[97103]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:29.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:29 smithi134 ceph-mon[130447]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:29.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:29 smithi192 ceph-mon[130806]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:31.347 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:31 smithi032 ceph-mon[97103]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:31.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:31 smithi134 ceph-mon[130447]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:31.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:31 smithi192 ceph-mon[130806]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 -- 172.21.15.32:0/568533262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578100940 msgr2=0x7fd578100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 --2- 172.21.15.32:0/568533262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578100940 0x7fd578100dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd568009990 tx=0x7fd568009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 -- 172.21.15.32:0/568533262 shutdown_connections 2023-08-11T15:45:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 --2- 172.21.15.32:0/568533262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd578101300 0x7fd57810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 --2- 172.21.15.32:0/568533262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd578106940 0x7fd578106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 --2- 172.21.15.32:0/568533262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578100940 0x7fd578100dc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 -- 172.21.15.32:0/568533262 >> 172.21.15.32:0/568533262 conn(0x7fd5780fc410 msgr2=0x7fd5780fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 -- 172.21.15.32:0/568533262 shutdown_connections 2023-08-11T15:45:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.075+0000 7fd57fddc700 1 -- 172.21.15.32:0/568533262 wait complete. 2023-08-11T15:45:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.076+0000 7fd57fddc700 1 Processor -- start 2023-08-11T15:45:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57fddc700 1 -- start start 2023-08-11T15:45:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57fddc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd578100940 0x7fd57819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57fddc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd578101300 0x7fd57819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57fddc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578106940 0x7fd578198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57fddc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd578199240 con 0x7fd578106940 2023-08-11T15:45:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57fddc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781993b0 con 0x7fd578100940 2023-08-11T15:45:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57fddc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd578199570 con 0x7fd578101300 2023-08-11T15:45:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57e379700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578106940 0x7fd578198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57d377700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd578101300 0x7fd57819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57e379700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578106940 0x7fd578198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44590/0 (socket says 172.21.15.32:44590) 2023-08-11T15:45:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57e379700 1 -- 172.21.15.32:0/1100004252 learned_addr learned my addr 172.21.15.32:0/1100004252 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.077+0000 7fd57db78700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd578100940 0x7fd57819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:32.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57e379700 1 -- 172.21.15.32:0/1100004252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd578101300 msgr2=0x7fd57819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57e379700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd578101300 0x7fd57819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57e379700 1 -- 172.21.15.32:0/1100004252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd578100940 msgr2=0x7fd57819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57e379700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd578100940 0x7fd57819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57e379700 1 -- 172.21.15.32:0/1100004252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680095f0 con 0x7fd578106940 2023-08-11T15:45:32.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57e379700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578106940 0x7fd578198a90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd57000ef30 tx=0x7fd57000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd56effd700 1 -- 172.21.15.32:0/1100004252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570018070 con 0x7fd578106940 2023-08-11T15:45:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5781997f0 con 0x7fd578106940 2023-08-11T15:45:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd56effd700 1 -- 172.21.15.32:0/1100004252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57000f040 con 0x7fd578106940 2023-08-11T15:45:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd56effd700 1 -- 172.21.15.32:0/1100004252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000cd00 con 0x7fd578106940 2023-08-11T15:45:32.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.078+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5781a5a20 con 0x7fd578106940 2023-08-11T15:45:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.079+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57804e620 con 0x7fd578106940 2023-08-11T15:45:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.081+0000 7fd56effd700 1 -- 172.21.15.32:0/1100004252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd570003a00 con 0x7fd578106940 2023-08-11T15:45:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.083+0000 7fd56effd700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd56409ffe0 0x7fd5640a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.083+0000 7fd56effd700 1 -- 172.21.15.32:0/1100004252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd5700cb2a0 con 0x7fd578106940 2023-08-11T15:45:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.083+0000 7fd57db78700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd56409ffe0 0x7fd5640a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.084+0000 7fd57db78700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd56409ffe0 0x7fd5640a24a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd574009be0 tx=0x7fd574009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:32.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.088+0000 7fd56effd700 1 -- 172.21.15.32:0/1100004252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd570094bf0 con 0x7fd578106940 2023-08-11T15:45:32.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.330+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd578110a80 con 0x7fd578106940 2023-08-11T15:45:32.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.331+0000 7fd56effd700 1 -- 172.21.15.32:0/1100004252 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd570098210 con 0x7fd578106940 2023-08-11T15:45:32.333 INFO:teuthology.orchestra.run.smithi032.stdout:111669149758 2023-08-11T15:45:32.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.333+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd56409ffe0 msgr2=0x7fd5640a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.333+0000 7fd57fddc700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd56409ffe0 0x7fd5640a24a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd574009be0 tx=0x7fd574009260 comp rx=0 tx=0).stop 2023-08-11T15:45:32.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.333+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578106940 msgr2=0x7fd578198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.333+0000 7fd57fddc700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578106940 0x7fd578198a90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd57000ef30 tx=0x7fd57000c3a0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.334+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 shutdown_connections 2023-08-11T15:45:32.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.334+0000 7fd57fddc700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd56409ffe0 0x7fd5640a24a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.334+0000 7fd57fddc700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd578106940 0x7fd578198a90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.334+0000 7fd57fddc700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd578100940 0x7fd57819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.334+0000 7fd57fddc700 1 --2- 172.21.15.32:0/1100004252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd578101300 0x7fd57819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.334+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 >> 172.21.15.32:0/1100004252 conn(0x7fd5780fc410 msgr2=0x7fd57810bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:32.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.335+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 shutdown_connections 2023-08-11T15:45:32.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.335+0000 7fd57fddc700 1 -- 172.21.15.32:0/1100004252 wait complete. 2023-08-11T15:45:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 -- 172.21.15.32:0/3614336178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938100340 msgr2=0x7f4938100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 --2- 172.21.15.32:0/3614336178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938100340 0x7f4938100720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4934009990 tx=0x7f4934009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 -- 172.21.15.32:0/3614336178 shutdown_connections 2023-08-11T15:45:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 --2- 172.21.15.32:0/3614336178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493810d3b0 0x7f493810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 --2- 172.21.15.32:0/3614336178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938100340 0x7f4938100720 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 --2- 172.21.15.32:0/3614336178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4938100c60 0x7f493810ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 -- 172.21.15.32:0/3614336178 >> 172.21.15.32:0/3614336178 conn(0x7f4938076920 msgr2=0x7f4938076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 -- 172.21.15.32:0/3614336178 shutdown_connections 2023-08-11T15:45:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.241+0000 7f493ff53700 1 -- 172.21.15.32:0/3614336178 wait complete. 2023-08-11T15:45:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.242+0000 7f493ff53700 1 Processor -- start 2023-08-11T15:45:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.243+0000 7f493ff53700 1 -- start start 2023-08-11T15:45:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.243+0000 7f493ff53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4938100340 0x7f4938198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.243+0000 7f493ff53700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4938100c60 0x7f4938199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.243+0000 7f493ff53700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493810d3b0 0x7f493819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.243+0000 7f493ff53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493819dcd0 con 0x7f4938100340 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.243+0000 7f493ff53700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493819de40 con 0x7f493810d3b0 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.243+0000 7f493ff53700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493819e000 con 0x7f4938100c60 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4938100340 0x7f4938198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4938100340 0x7f4938198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44602/0 (socket says 172.21.15.32:44602) 2023-08-11T15:45:32.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 -- 172.21.15.32:0/2644821751 learned_addr learned my addr 172.21.15.32:0/2644821751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493e4f0700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493810d3b0 0x7f493819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 -- 172.21.15.32:0/2644821751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4938100c60 msgr2=0x7f4938199250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T15:45:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4938100c60 0x7f4938199250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 -- 172.21.15.32:0/2644821751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493810d3b0 msgr2=0x7f493819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493810d3b0 0x7f493819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 -- 172.21.15.32:0/2644821751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009520 con 0x7f4938100340 2023-08-11T15:45:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.244+0000 7f493dcef700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4938100340 0x7f4938198d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f493400b680 tx=0x7f4934009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.245+0000 7f492effd700 1 -- 172.21.15.32:0/2644821751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493401d070 con 0x7f4938100340 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.245+0000 7f492effd700 1 -- 172.21.15.32:0/2644821751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4934021410 con 0x7f4938100340 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.245+0000 7f492effd700 1 -- 172.21.15.32:0/2644821751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934017650 con 0x7f4938100340 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.245+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49340095f0 con 0x7f4938100340 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.245+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493819e4b0 con 0x7f4938100340 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.246+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493804e620 con 0x7f4938100340 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.249+0000 7f492effd700 1 -- 172.21.15.32:0/2644821751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f4934021580 con 0x7f4938100340 2023-08-11T15:45:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.251+0000 7f492effd700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f492409ffe0 0x7f49240a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:32.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.251+0000 7f492effd700 1 -- 172.21.15.32:0/2644821751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f49340ccb50 con 0x7f4938100340 2023-08-11T15:45:32.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.251+0000 7f493d4ee700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f492409ffe0 0x7f49240a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:32.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.252+0000 7f493d4ee700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f492409ffe0 0x7f49240a24a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f493819a330 tx=0x7f4928009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:32.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.258+0000 7f492effd700 1 -- 172.21.15.32:0/2644821751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f49340964a0 con 0x7f4938100340 2023-08-11T15:45:32.670 INFO:teuthology.orchestra.run.smithi032.stdout:55834574919 2023-08-11T15:45:32.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.511+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4938112b40 con 0x7f4938100340 2023-08-11T15:45:32.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.511+0000 7f492effd700 1 -- 172.21.15.32:0/2644821751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4934099ac0 con 0x7f4938100340 2023-08-11T15:45:32.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.514+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f492409ffe0 msgr2=0x7f49240a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.514+0000 7f493ff53700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f492409ffe0 0x7f49240a24a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f493819a330 tx=0x7f4928009260 comp rx=0 tx=0).stop 2023-08-11T15:45:32.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.514+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4938100340 msgr2=0x7f4938198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:32.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.514+0000 7f493ff53700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4938100340 0x7f4938198d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f493400b680 tx=0x7f4934009580 comp rx=0 tx=0).stop 2023-08-11T15:45:32.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 shutdown_connections 2023-08-11T15:45:32.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f492409ffe0 0x7f49240a24a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4938100340 0x7f4938198d10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493810d3b0 0x7f493819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 --2- 172.21.15.32:0/2644821751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4938100c60 0x7f4938199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:32.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 >> 172.21.15.32:0/2644821751 conn(0x7f4938076920 msgr2=0x7f493810e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:32.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 shutdown_connections 2023-08-11T15:45:32.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:32.515+0000 7f493ff53700 1 -- 172.21.15.32:0/2644821751 wait complete. 2023-08-11T15:45:33.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:32 smithi134 ceph-mon[130447]: pgmap v309: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:33.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:32 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1100004252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-11T15:45:33.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:32 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2644821751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-11T15:45:33.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:32 smithi032 ceph-mon[97103]: pgmap v309: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:33.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:32 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1100004252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-11T15:45:33.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:32 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2644821751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-11T15:45:33.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:32 smithi192 ceph-mon[130806]: pgmap v309: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:33.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:32 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1100004252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-11T15:45:33.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:32 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2644821751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-11T15:45:33.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149758 for osd.3 2023-08-11T15:45:33.983 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 -- 172.21.15.32:0/2127108785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1858105390 msgr2=0x7f1858105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 --2- 172.21.15.32:0/2127108785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1858105390 0x7f1858105810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f184c0099c0 tx=0x7f184c009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 -- 172.21.15.32:0/2127108785 shutdown_connections 2023-08-11T15:45:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 --2- 172.21.15.32:0/2127108785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1858105d50 0x7f185810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 --2- 172.21.15.32:0/2127108785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18580688e0 0x7f1858068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 --2- 172.21.15.32:0/2127108785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1858105390 0x7f1858105810 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 -- 172.21.15.32:0/2127108785 >> 172.21.15.32:0/2127108785 conn(0x7f1858076920 msgr2=0x7f1858076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 -- 172.21.15.32:0/2127108785 shutdown_connections 2023-08-11T15:45:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.475+0000 7f185dba4700 1 -- 172.21.15.32:0/2127108785 wait complete. 2023-08-11T15:45:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.477+0000 7f185dba4700 1 Processor -- start 2023-08-11T15:45:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.477+0000 7f185dba4700 1 -- start start 2023-08-11T15:45:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f185dba4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580688e0 0x7f185819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f185dba4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1858105390 0x7f185819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f185dba4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1858105d50 0x7f18581968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f185dba4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18581970a0 con 0x7f18580688e0 2023-08-11T15:45:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f185dba4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1858197210 con 0x7f1858105390 2023-08-11T15:45:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f185dba4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18581973d0 con 0x7f1858105d50 2023-08-11T15:45:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f18577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580688e0 0x7f185819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f18577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580688e0 0x7f185819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44632/0 (socket says 172.21.15.32:44632) 2023-08-11T15:45:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f18577fe700 1 -- 172.21.15.32:0/1121313142 learned_addr learned my addr 172.21.15.32:0/1121313142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f1856ffd700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1858105390 0x7f185819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.478+0000 7f1857fff700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1858105d50 0x7f18581968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f18577fe700 1 -- 172.21.15.32:0/1121313142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1858105d50 msgr2=0x7f18581968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f18577fe700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1858105d50 0x7f18581968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f18577fe700 1 -- 172.21.15.32:0/1121313142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1858105390 msgr2=0x7f185819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f18577fe700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1858105390 0x7f185819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f18577fe700 1 -- 172.21.15.32:0/1121313142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1850009c40 con 0x7f18580688e0 2023-08-11T15:45:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f1857fff700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1858105d50 0x7f18581968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f18577fe700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580688e0 0x7f185819c900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f184800e9a0 tx=0x7f184800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.479+0000 7f1854ff9700 1 -- 172.21.15.32:0/1121313142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848009790 con 0x7f18580688e0 2023-08-11T15:45:34.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.480+0000 7f1854ff9700 1 -- 172.21.15.32:0/1121313142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1848010430 con 0x7f18580688e0 2023-08-11T15:45:34.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.480+0000 7f1854ff9700 1 -- 172.21.15.32:0/1121313142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848018680 con 0x7f18580688e0 2023-08-11T15:45:34.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.480+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c0095f0 con 0x7f18580688e0 2023-08-11T15:45:34.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.480+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18581a3880 con 0x7f18580688e0 2023-08-11T15:45:34.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.481+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185804e620 con 0x7f18580688e0 2023-08-11T15:45:34.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.485+0000 7f1854ff9700 1 -- 172.21.15.32:0/1121313142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f18480105a0 con 0x7f18580688e0 2023-08-11T15:45:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.487+0000 7f1854ff9700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f18400a0060 0x7f18400a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.487+0000 7f1854ff9700 1 -- 172.21.15.32:0/1121313142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1848014070 con 0x7f18580688e0 2023-08-11T15:45:34.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.488+0000 7f1856ffd700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f18400a0060 0x7f18400a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:34.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.488+0000 7f1856ffd700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f18400a0060 0x7f18400a2520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:34.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.495+0000 7f1854ff9700 1 -- 172.21.15.32:0/1121313142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1848095170 con 0x7f18580688e0 2023-08-11T15:45:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 -- 172.21.15.32:0/3989789132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970106940 msgr2=0x7f4970106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 --2- 172.21.15.32:0/3989789132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970106940 0x7f4970106d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f496c00b0a0 tx=0x7f496c00b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 -- 172.21.15.32:0/3989789132 shutdown_connections 2023-08-11T15:45:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 --2- 172.21.15.32:0/3989789132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970106940 0x7f4970106d20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 --2- 172.21.15.32:0/3989789132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4970101300 0x7f497010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 --2- 172.21.15.32:0/3989789132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970100940 0x7f4970100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 -- 172.21.15.32:0/3989789132 >> 172.21.15.32:0/3989789132 conn(0x7f49700fc410 msgr2=0x7f49700fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 -- 172.21.15.32:0/3989789132 shutdown_connections 2023-08-11T15:45:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.439+0000 7f4977cce700 1 -- 172.21.15.32:0/3989789132 wait complete. 2023-08-11T15:45:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.440+0000 7f4977cce700 1 Processor -- start 2023-08-11T15:45:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.441+0000 7f4977cce700 1 -- start start 2023-08-11T15:45:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.441+0000 7f4977cce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970100940 0x7f4970198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.441+0000 7f4977cce700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970101300 0x7f49701992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.441+0000 7f4977cce700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4970106940 0x7f497019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.441+0000 7f4977cce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497019dd70 con 0x7f4970100940 2023-08-11T15:45:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.441+0000 7f4977cce700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497019dee0 con 0x7f4970106940 2023-08-11T15:45:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.441+0000 7f4977cce700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497019e0a0 con 0x7f4970101300 2023-08-11T15:45:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970101300 0x7f49701992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970101300 0x7f49701992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:48712/0 (socket says 172.21.15.32:48712) 2023-08-11T15:45:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 -- 172.21.15.32:0/4287724796 learned_addr learned my addr 172.21.15.32:0/4287724796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 -- 172.21.15.32:0/4287724796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4970106940 msgr2=0x7f497019d760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:45:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975a6a700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970100940 0x7f4970198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4970106940 0x7f497019d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 -- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970100940 msgr2=0x7f4970198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970100940 0x7f4970198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975269700 1 -- 172.21.15.32:0/4287724796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009d00 con 0x7f4970101300 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.442+0000 7f4975a6a700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970100940 0x7f4970198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.443+0000 7f4975269700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970101300 0x7f49701992f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f496000e9a0 tx=0x7f496000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.443+0000 7f4966ffd700 1 -- 172.21.15.32:0/4287724796 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960009790 con 0x7f4970101300 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.443+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497019e380 con 0x7f4970101300 2023-08-11T15:45:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.443+0000 7f4966ffd700 1 -- 172.21.15.32:0/4287724796 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f496000cbd0 con 0x7f4970101300 2023-08-11T15:45:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.443+0000 7f4966ffd700 1 -- 172.21.15.32:0/4287724796 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960018680 con 0x7f4970101300 2023-08-11T15:45:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.443+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49701a5be0 con 0x7f4970101300 2023-08-11T15:45:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.444+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497004e620 con 0x7f4970101300 2023-08-11T15:45:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.446+0000 7f4966ffd700 1 -- 172.21.15.32:0/4287724796 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f496000cd40 con 0x7f4970101300 2023-08-11T15:45:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.448+0000 7f4966ffd700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f495c09ffe0 0x7f495c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.448+0000 7f4966ffd700 1 -- 172.21.15.32:0/4287724796 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4960014070 con 0x7f4970101300 2023-08-11T15:45:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.449+0000 7f4975a6a700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f495c09ffe0 0x7f495c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.449+0000 7f4975a6a700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f495c09ffe0 0x7f495c0a24a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f496c00b070 tx=0x7f496c00b720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.450+0000 7f4966ffd700 1 -- 172.21.15.32:0/4287724796 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f49600954e0 con 0x7f4970101300 2023-08-11T15:45:34.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.689+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4970110a80 con 0x7f4970101300 2023-08-11T15:45:34.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.689+0000 7f4966ffd700 1 -- 172.21.15.32:0/4287724796 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4960098b00 con 0x7f4970101300 2023-08-11T15:45:34.691 INFO:teuthology.orchestra.run.smithi032.stdout:34359738444 2023-08-11T15:45:34.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f495c09ffe0 msgr2=0x7f495c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f495c09ffe0 0x7f495c0a24a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f496c00b070 tx=0x7f496c00b720 comp rx=0 tx=0).stop 2023-08-11T15:45:34.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970101300 msgr2=0x7f49701992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970101300 0x7f49701992f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f496000e9a0 tx=0x7f496000ed60 comp rx=0 tx=0).stop 2023-08-11T15:45:34.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 shutdown_connections 2023-08-11T15:45:34.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f495c09ffe0 0x7f495c0a24a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4970100940 0x7f4970198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4970106940 0x7f497019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 --2- 172.21.15.32:0/4287724796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4970101300 0x7f49701992f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 >> 172.21.15.32:0/4287724796 conn(0x7f49700fc410 msgr2=0x7f497010bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:34.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 shutdown_connections 2023-08-11T15:45:34.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.692+0000 7f4977cce700 1 -- 172.21.15.32:0/4287724796 wait complete. 2023-08-11T15:45:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.762+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f1858197e20 con 0x7f18580688e0 2023-08-11T15:45:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.762+0000 7f1854ff9700 1 -- 172.21.15.32:0/1121313142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1848098790 con 0x7f18580688e0 2023-08-11T15:45:34.764 INFO:teuthology.orchestra.run.smithi032.stdout:309237645342 2023-08-11T15:45:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f18400a0060 msgr2=0x7f18400a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f18400a0060 0x7f18400a2520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580688e0 msgr2=0x7f185819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580688e0 0x7f185819c900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f184800e9a0 tx=0x7f184800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 shutdown_connections 2023-08-11T15:45:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f18400a0060 0x7f18400a2520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18580688e0 0x7f185819c900 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1858105d50 0x7f18581968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 --2- 172.21.15.32:0/1121313142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1858105390 0x7f185819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 >> 172.21.15.32:0/1121313142 conn(0x7f1858076920 msgr2=0x7f18580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:34.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 shutdown_connections 2023-08-11T15:45:34.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:34.765+0000 7f185dba4700 1 -- 172.21.15.32:0/1121313142 wait complete. 2023-08-11T15:45:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.074+0000 7fb4f4eec700 1 -- 172.21.15.32:0/807672889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 msgr2=0x7fb4f0106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.074+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/807672889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0106d20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009990 tx=0x7fb4e0009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.075+0000 7fb4f4eec700 1 -- 172.21.15.32:0/807672889 shutdown_connections 2023-08-11T15:45:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.075+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/807672889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0106d20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.075+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/807672889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f0101300 0x7fb4f010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.075+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/807672889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0100940 0x7fb4f0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.075+0000 7fb4f4eec700 1 -- 172.21.15.32:0/807672889 >> 172.21.15.32:0/807672889 conn(0x7fb4f00fc410 msgr2=0x7fb4f00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:35.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.075+0000 7fb4f4eec700 1 -- 172.21.15.32:0/807672889 shutdown_connections 2023-08-11T15:45:35.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.075+0000 7fb4f4eec700 1 -- 172.21.15.32:0/807672889 wait complete. 2023-08-11T15:45:35.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.076+0000 7fb4f4eec700 1 Processor -- start 2023-08-11T15:45:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.076+0000 7fb4f4eec700 1 -- start start 2023-08-11T15:45:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.077+0000 7fb4f4eec700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f0100940 0x7fb4f019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.077+0000 7fb4f4eec700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0101300 0x7fb4f019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.077+0000 7fb4f4eec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.077+0000 7fb4f4eec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f01990a0 con 0x7fb4f0106940 2023-08-11T15:45:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.077+0000 7fb4f4eec700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0199210 con 0x7fb4f0100940 2023-08-11T15:45:35.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.077+0000 7fb4f4eec700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f01993d0 con 0x7fb4f0101300 2023-08-11T15:45:35.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4eed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0101300 0x7fb4f019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4e659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f0100940 0x7fb4f019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4eed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46098/0 (socket says 172.21.15.32:46098) 2023-08-11T15:45:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4eed9d700 1 -- 172.21.15.32:0/21104756 learned_addr learned my addr 172.21.15.32:0/21104756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4e659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f0100940 0x7fb4f019e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35382/0 (socket says 172.21.15.32:35382) 2023-08-11T15:45:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0101300 0x7fb4f019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46850/0 (socket says 172.21.15.32:46850) 2023-08-11T15:45:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 -- 172.21.15.32:0/21104756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f0100940 msgr2=0x7fb4f019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f0100940 0x7fb4f019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 -- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 msgr2=0x7fb4f0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 -- 172.21.15.32:0/21104756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00095f0 con 0x7fb4f0101300 2023-08-11T15:45:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4eed9d700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.078+0000 7fb4ee59c700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0101300 0x7fb4f019eef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800d750 tx=0x7fb4d800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.079+0000 7fb4e7fff700 1 -- 172.21.15.32:0/21104756 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8004020 con 0x7fb4f0101300 2023-08-11T15:45:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.079+0000 7fb4e7fff700 1 -- 172.21.15.32:0/21104756 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d8004830 con 0x7fb4f0101300 2023-08-11T15:45:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.079+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f01996b0 con 0x7fb4f0101300 2023-08-11T15:45:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.079+0000 7fb4e7fff700 1 -- 172.21.15.32:0/21104756 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8005550 con 0x7fb4f0101300 2023-08-11T15:45:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.079+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f01a59a0 con 0x7fb4f0101300 2023-08-11T15:45:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.080+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f004e620 con 0x7fb4f0101300 2023-08-11T15:45:35.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.083+0000 7fb4e7fff700 1 -- 172.21.15.32:0/21104756 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb4d8004ca0 con 0x7fb4f0101300 2023-08-11T15:45:35.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.085+0000 7fb4e7fff700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4dc09ffe0 0x7fb4dc0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.085+0000 7fb4e7fff700 1 -- 172.21.15.32:0/21104756 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb4d80cb2e0 con 0x7fb4f0101300 2023-08-11T15:45:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.085+0000 7fb4e659c700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4dc09ffe0 0x7fb4dc0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.085+0000 7fb4e7fff700 1 -- 172.21.15.32:0/21104756 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb4d8094ca0 con 0x7fb4f0101300 2023-08-11T15:45:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.086+0000 7fb4e659c700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4dc09ffe0 0x7fb4dc0a24a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000b2e0 tx=0x7fb4e0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:35.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574919 for osd.1 2023-08-11T15:45:35.093 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:35.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.322+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb4f01109e0 con 0x7fb4f0101300 2023-08-11T15:45:35.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.323+0000 7fb4e7fff700 1 -- 172.21.15.32:0/21104756 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4d80982c0 con 0x7fb4f0101300 2023-08-11T15:45:35.325 INFO:teuthology.orchestra.run.smithi032.stdout:167503724598 2023-08-11T15:45:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.325+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4dc09ffe0 msgr2=0x7fb4dc0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.325+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4dc09ffe0 0x7fb4dc0a24a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000b2e0 tx=0x7fb4e0005fd0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.325+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0101300 msgr2=0x7fb4f019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0101300 0x7fb4f019eef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800d750 tx=0x7fb4d800da60 comp rx=0 tx=0).stop 2023-08-11T15:45:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 shutdown_connections 2023-08-11T15:45:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb4dc09ffe0 0x7fb4dc0a24a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f0106940 0x7fb4f0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f0100940 0x7fb4f019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 --2- 172.21.15.32:0/21104756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f0101300 0x7fb4f019eef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 >> 172.21.15.32:0/21104756 conn(0x7fb4f00fc410 msgr2=0x7fb4f010bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:35.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 shutdown_connections 2023-08-11T15:45:35.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.326+0000 7fb4f4eec700 1 -- 172.21.15.32:0/21104756 wait complete. 2023-08-11T15:45:35.351 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:35 smithi032 ceph-mon[97103]: pgmap v310: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:35.351 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:35 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4287724796' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-11T15:45:35.351 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:35 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1121313142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-11T15:45:35.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:35 smithi134 ceph-mon[130447]: pgmap v310: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:35.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:35 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4287724796' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-11T15:45:35.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:35 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1121313142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-11T15:45:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:35 smithi192 ceph-mon[130806]: pgmap v310: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:35 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4287724796' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-11T15:45:35.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:35 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1121313142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-11T15:45:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.636+0000 7fab42417700 1 -- 172.21.15.32:0/4286045277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 msgr2=0x7fab3c10cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.636+0000 7fab42417700 1 --2- 172.21.15.32:0/4286045277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c10cc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab30009990 tx=0x7fab30009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.636+0000 7fab42417700 1 -- 172.21.15.32:0/4286045277 shutdown_connections 2023-08-11T15:45:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.636+0000 7fab42417700 1 --2- 172.21.15.32:0/4286045277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab3c10d1d0 0x7fab3c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.636+0000 7fab42417700 1 --2- 172.21.15.32:0/4286045277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab3c072d50 0x7fab3c073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.636+0000 7fab42417700 1 --2- 172.21.15.32:0/4286045277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c10cc90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.636+0000 7fab42417700 1 -- 172.21.15.32:0/4286045277 >> 172.21.15.32:0/4286045277 conn(0x7fab3c0fc480 msgr2=0x7fab3c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.637+0000 7fab42417700 1 -- 172.21.15.32:0/4286045277 shutdown_connections 2023-08-11T15:45:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.637+0000 7fab42417700 1 -- 172.21.15.32:0/4286045277 wait complete. 2023-08-11T15:45:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.638+0000 7fab42417700 1 Processor -- start 2023-08-11T15:45:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.638+0000 7fab42417700 1 -- start start 2023-08-11T15:45:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab42417700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab42417700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab3c10d1d0 0x7fab3c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab42417700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab3c19d770 0x7fab3c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab42417700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19e160 con 0x7fab3c19d770 2023-08-11T15:45:35.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab42417700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19e2d0 con 0x7fab3c10d1d0 2023-08-11T15:45:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab42417700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19e490 con 0x7fab3c073670 2023-08-11T15:45:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab3bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab3bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46880/0 (socket says 172.21.15.32:46880) 2023-08-11T15:45:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab3bfff700 1 -- 172.21.15.32:0/340219162 learned_addr learned my addr 172.21.15.32:0/340219162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab409b4700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab3c19d770 0x7fab3c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.639+0000 7fab3b7fe700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab3c10d1d0 0x7fab3c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab3bfff700 1 -- 172.21.15.32:0/340219162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab3c10d1d0 msgr2=0x7fab3c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab3bfff700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab3c10d1d0 0x7fab3c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab3bfff700 1 -- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab3c19d770 msgr2=0x7fab3c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab3bfff700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab3c19d770 0x7fab3c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab3bfff700 1 -- 172.21.15.32:0/340219162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab300095f0 con 0x7fab3c073670 2023-08-11T15:45:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab3b7fe700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab3c10d1d0 0x7fab3c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:45:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab3bfff700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c198dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00d900 tx=0x7fab2c00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab397fa700 1 -- 172.21.15.32:0/340219162 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c009930 con 0x7fab3c073670 2023-08-11T15:45:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c1a5a20 con 0x7fab3c073670 2023-08-11T15:45:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab397fa700 1 -- 172.21.15.32:0/340219162 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab2c00b5c0 con 0x7fab3c073670 2023-08-11T15:45:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab397fa700 1 -- 172.21.15.32:0/340219162 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00f3c0 con 0x7fab3c073670 2023-08-11T15:45:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.640+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c1a5f20 con 0x7fab3c073670 2023-08-11T15:45:35.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.641+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c107af0 con 0x7fab3c073670 2023-08-11T15:45:35.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.644+0000 7fab397fa700 1 -- 172.21.15.32:0/340219162 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fab2c009a90 con 0x7fab3c073670 2023-08-11T15:45:35.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.645+0000 7fab397fa700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab240a0020 0x7fab240a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.645+0000 7fab397fa700 1 -- 172.21.15.32:0/340219162 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fab2c0cb1d0 con 0x7fab3c073670 2023-08-11T15:45:35.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.645+0000 7fab3b7fe700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab240a0020 0x7fab240a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.646+0000 7fab3b7fe700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab240a0020 0x7fab240a24e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab3000b5c0 tx=0x7fab30005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.647+0000 7fab397fa700 1 -- 172.21.15.32:0/340219162 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fab2c094a90 con 0x7fab3c073670 2023-08-11T15:45:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:35.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738444 for osd.0 2023-08-11T15:45:35.853 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.871+0000 7f815dc28700 1 -- 172.21.15.32:0/2398053042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8158102e30 msgr2=0x7f815810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.871+0000 7f815dc28700 1 --2- 172.21.15.32:0/2398053042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8158102e30 0x7f815810d8e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f815000b0a0 tx=0x7f815000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.872+0000 7f815dc28700 1 -- 172.21.15.32:0/2398053042 shutdown_connections 2023-08-11T15:45:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.872+0000 7f815dc28700 1 --2- 172.21.15.32:0/2398053042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8158102e30 0x7f815810d8e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.872+0000 7f815dc28700 1 --2- 172.21.15.32:0/2398053042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81580690a0 0x7f8158069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.872+0000 7f815dc28700 1 --2- 172.21.15.32:0/2398053042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8158102470 0x7f81581028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.872+0000 7f815dc28700 1 -- 172.21.15.32:0/2398053042 >> 172.21.15.32:0/2398053042 conn(0x7f8158076910 msgr2=0x7f8158076d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:35.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.873+0000 7f815dc28700 1 -- 172.21.15.32:0/2398053042 shutdown_connections 2023-08-11T15:45:35.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.874+0000 7f815dc28700 1 -- 172.21.15.32:0/2398053042 wait complete. 2023-08-11T15:45:35.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.875+0000 7f815dc28700 1 Processor -- start 2023-08-11T15:45:35.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.876+0000 7f815dc28700 1 -- start start 2023-08-11T15:45:35.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.876+0000 7f815dc28700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8158102470 0x7f8158198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.876+0000 7f815dc28700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8158102e30 0x7f8158199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.876+0000 7f815dc28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815819d7a0 0x7f815819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.876+0000 7f815dc28700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819e190 con 0x7f815819d7a0 2023-08-11T15:45:35.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.876+0000 7f815dc28700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819e300 con 0x7f8158102e30 2023-08-11T15:45:35.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.876+0000 7f815dc28700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819e4c0 con 0x7f8158102470 2023-08-11T15:45:35.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815819d7a0 0x7f815819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815819d7a0 0x7f815819dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46136/0 (socket says 172.21.15.32:46136) 2023-08-11T15:45:35.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 -- 172.21.15.32:0/1948965138 learned_addr learned my addr 172.21.15.32:0/1948965138 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:35.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f81577fe700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8158102470 0x7f8158198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8156ffd700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8158102e30 0x7f8158199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 -- 172.21.15.32:0/1948965138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8158102470 msgr2=0x7f8158198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8158102470 0x7f8158198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 -- 172.21.15.32:0/1948965138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8158102e30 msgr2=0x7f8158199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8158102e30 0x7f8158199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.877+0000 7f8157fff700 1 -- 172.21.15.32:0/1948965138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8150009d00 con 0x7f815819d7a0 2023-08-11T15:45:35.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.878+0000 7f8157fff700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815819d7a0 0x7f815819dc20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8150009fd0 tx=0x7f8150007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:35.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.878+0000 7f8154ff9700 1 -- 172.21.15.32:0/1948965138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815000e040 con 0x7f815819d7a0 2023-08-11T15:45:35.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.879+0000 7f8154ff9700 1 -- 172.21.15.32:0/1948965138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8150003c00 con 0x7f815819d7a0 2023-08-11T15:45:35.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.879+0000 7f8154ff9700 1 -- 172.21.15.32:0/1948965138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81500054d0 con 0x7f815819d7a0 2023-08-11T15:45:35.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.879+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81581a5a50 con 0x7f815819d7a0 2023-08-11T15:45:35.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.879+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81581a5ef0 con 0x7f815819d7a0 2023-08-11T15:45:35.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.880+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815804e620 con 0x7f815819d7a0 2023-08-11T15:45:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.884+0000 7f8154ff9700 1 -- 172.21.15.32:0/1948965138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f8150005630 con 0x7f815819d7a0 2023-08-11T15:45:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.886+0000 7f8154ff9700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81400a0060 0x7f81400a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.886+0000 7f8154ff9700 1 -- 172.21.15.32:0/1948965138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81500cd370 con 0x7f815819d7a0 2023-08-11T15:45:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.887+0000 7f81577fe700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81400a0060 0x7f81400a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:35.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.887+0000 7f81577fe700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81400a0060 0x7f81400a2520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8148005950 tx=0x7f814800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.889+0000 7f8154ff9700 1 -- 172.21.15.32:0/1948965138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8150096db0 con 0x7f815819d7a0 2023-08-11T15:45:35.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.928+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fab3c04e620 con 0x7fab3c073670 2023-08-11T15:45:35.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.928+0000 7fab397fa700 1 -- 172.21.15.32:0/340219162 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab2c021090 con 0x7fab3c073670 2023-08-11T15:45:35.930 INFO:teuthology.orchestra.run.smithi032.stdout:287762808868 2023-08-11T15:45:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.932+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab240a0020 msgr2=0x7fab240a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.932+0000 7fab42417700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab240a0020 0x7fab240a24e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab3000b5c0 tx=0x7fab30005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.932+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 msgr2=0x7fab3c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.932+0000 7fab42417700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c198dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00d900 tx=0x7fab2c00dc10 comp rx=0 tx=0).stop 2023-08-11T15:45:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 shutdown_connections 2023-08-11T15:45:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fab240a0020 0x7fab240a24e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab3c19d770 0x7fab3c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab3c073670 0x7fab3c198dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 --2- 172.21.15.32:0/340219162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab3c10d1d0 0x7fab3c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 >> 172.21.15.32:0/340219162 conn(0x7fab3c0fc480 msgr2=0x7fab3c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 shutdown_connections 2023-08-11T15:45:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:35.933+0000 7fab42417700 1 -- 172.21.15.32:0/340219162 wait complete. 2023-08-11T15:45:36.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.138+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f8158110aa0 con 0x7f815819d7a0 2023-08-11T15:45:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.138+0000 7f8154ff9700 1 -- 172.21.15.32:0/1948965138 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f815009a3d0 con 0x7f815819d7a0 2023-08-11T15:45:36.140 INFO:teuthology.orchestra.run.smithi032.stdout:339302416411 2023-08-11T15:45:36.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81400a0060 msgr2=0x7f81400a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:36.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81400a0060 0x7f81400a2520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8148005950 tx=0x7f814800b390 comp rx=0 tx=0).stop 2023-08-11T15:45:36.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815819d7a0 msgr2=0x7f815819dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:36.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815819d7a0 0x7f815819dc20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8150009fd0 tx=0x7f8150007af0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 shutdown_connections 2023-08-11T15:45:36.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81400a0060 0x7f81400a2520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f815819d7a0 0x7f815819dc20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8158102470 0x7f8158198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 --2- 172.21.15.32:0/1948965138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8158102e30 0x7f8158199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 >> 172.21.15.32:0/1948965138 conn(0x7f8158076910 msgr2=0x7f81580fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:36.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 shutdown_connections 2023-08-11T15:45:36.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.141+0000 7f815dc28700 1 -- 172.21.15.32:0/1948965138 wait complete. 2023-08-11T15:45:36.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:36 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/21104756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-11T15:45:36.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:36.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:36 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:36.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:36 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/340219162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-11T15:45:36.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:36 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1948965138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-11T15:45:36.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:36 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/21104756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-11T15:45:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:36 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:36 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/340219162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-11T15:45:36.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:36 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1948965138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-11T15:45:36.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645340 got 309237645342 for osd.10 2023-08-11T15:45:36.539 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:36 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/21104756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-11T15:45:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:36 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:36 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/340219162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-11T15:45:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:36 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1948965138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-11T15:45:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.102+0000 7fa2bf052700 1 -- 172.21.15.32:0/83261835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8101300 msgr2=0x7fa2b810d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.102+0000 7fa2bf052700 1 --2- 172.21.15.32:0/83261835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8101300 0x7fa2b810d810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000b0a0 tx=0x7fa2b000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.102+0000 7fa2bf052700 1 -- 172.21.15.32:0/83261835 shutdown_connections 2023-08-11T15:45:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.102+0000 7fa2bf052700 1 --2- 172.21.15.32:0/83261835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b8106940 0x7fa2b8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.102+0000 7fa2bf052700 1 --2- 172.21.15.32:0/83261835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8101300 0x7fa2b810d810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.102+0000 7fa2bf052700 1 --2- 172.21.15.32:0/83261835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2b8100940 0x7fa2b8100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.102+0000 7fa2bf052700 1 -- 172.21.15.32:0/83261835 >> 172.21.15.32:0/83261835 conn(0x7fa2b80fc410 msgr2=0x7fa2b80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.103+0000 7fa2bf052700 1 -- 172.21.15.32:0/83261835 shutdown_connections 2023-08-11T15:45:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.103+0000 7fa2bf052700 1 -- 172.21.15.32:0/83261835 wait complete. 2023-08-11T15:45:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 Processor -- start 2023-08-11T15:45:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 -- start start 2023-08-11T15:45:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8100940 0x7fa2b8198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2b8101300 0x7fa2b8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b8106940 0x7fa2b819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b819dd10 con 0x7fa2b8106940 2023-08-11T15:45:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b819de80 con 0x7fa2b8101300 2023-08-11T15:45:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.104+0000 7fa2bf052700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b819e040 con 0x7fa2b8100940 2023-08-11T15:45:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bd5ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b8106940 0x7fa2b819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8100940 0x7fa2b8198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8100940 0x7fa2b8198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46912/0 (socket says 172.21.15.32:46912) 2023-08-11T15:45:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 -- 172.21.15.32:0/799910699 learned_addr learned my addr 172.21.15.32:0/799910699 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2b7fff700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2b8101300 0x7fa2b8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bd5ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b8106940 0x7fa2b819d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46166/0 (socket says 172.21.15.32:46166) 2023-08-11T15:45:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 -- 172.21.15.32:0/799910699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2b8101300 msgr2=0x7fa2b8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2b8101300 0x7fa2b8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 -- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b8106940 msgr2=0x7fa2b819d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b8106940 0x7fa2b819d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 -- 172.21.15.32:0/799910699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b0009d00 con 0x7fa2b8100940 2023-08-11T15:45:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.105+0000 7fa2bcdee700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8100940 0x7fa2b8198d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e9a0 tx=0x7fa2ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.106+0000 7fa2b5ffb700 1 -- 172.21.15.32:0/799910699 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac009790 con 0x7fa2b8100940 2023-08-11T15:45:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.106+0000 7fa2b5ffb700 1 -- 172.21.15.32:0/799910699 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2ac010430 con 0x7fa2b8100940 2023-08-11T15:45:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.106+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b819e320 con 0x7fa2b8100940 2023-08-11T15:45:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.106+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b81a5b80 con 0x7fa2b8100940 2023-08-11T15:45:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.106+0000 7fa2b5ffb700 1 -- 172.21.15.32:0/799910699 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac018680 con 0x7fa2b8100940 2023-08-11T15:45:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.107+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c005320 con 0x7fa2b8100940 2023-08-11T15:45:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.109+0000 7fa2b5ffb700 1 -- 172.21.15.32:0/799910699 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa2ac00ca10 con 0x7fa2b8100940 2023-08-11T15:45:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.112+0000 7fa2b5ffb700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa2a80a0060 0x7fa2a80a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.112+0000 7fa2b5ffb700 1 -- 172.21.15.32:0/799910699 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa2ac014070 con 0x7fa2b8100940 2023-08-11T15:45:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.112+0000 7fa2b7fff700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa2a80a0060 0x7fa2a80a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.112+0000 7fa2b5ffb700 1 -- 172.21.15.32:0/799910699 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa2ac0950c0 con 0x7fa2b8100940 2023-08-11T15:45:36.895 INFO:teuthology.orchestra.run.smithi032.stdout:223338299437 2023-08-11T15:45:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.113+0000 7fa2b7fff700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa2a80a0060 0x7fa2a80a2520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2b819a370 tx=0x7fa2a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.348+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa29c005190 con 0x7fa2b8100940 2023-08-11T15:45:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.348+0000 7fa2b5ffb700 1 -- 172.21.15.32:0/799910699 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2ac0986e0 con 0x7fa2b8100940 2023-08-11T15:45:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa2a80a0060 msgr2=0x7fa2a80a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa2a80a0060 0x7fa2a80a2520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2b819a370 tx=0x7fa2a4009260 comp rx=0 tx=0).stop 2023-08-11T15:45:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8100940 msgr2=0x7fa2b8198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8100940 0x7fa2b8198d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e9a0 tx=0x7fa2ac00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 shutdown_connections 2023-08-11T15:45:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa2a80a0060 0x7fa2a80a2520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b8106940 0x7fa2b819d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b8100940 0x7fa2b8198d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 --2- 172.21.15.32:0/799910699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2b8101300 0x7fa2b8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.351+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 >> 172.21.15.32:0/799910699 conn(0x7fa2b80fc410 msgr2=0x7fa2b80fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.352+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 shutdown_connections 2023-08-11T15:45:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:36.352+0000 7fa2bf052700 1 -- 172.21.15.32:0/799910699 wait complete. 2023-08-11T15:45:37.118 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:37.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416409 got 339302416411 for osd.11 2023-08-11T15:45:37.263 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:37.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724598 for osd.5 2023-08-11T15:45:37.362 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:37.425 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:37 smithi032 ceph-mon[97103]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:37.426 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:37 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/799910699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-11T15:45:37.457 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808868 for osd.9 2023-08-11T15:45:37.457 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:37.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:37 smithi134 ceph-mon[130447]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:37.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:37 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/799910699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-11T15:45:37.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:37 smithi192 ceph-mon[130806]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:37.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:37 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/799910699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-11T15:45:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:38.279 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:38 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:45:38.280 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:38 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:45:38.280 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:38 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.363+0000 7fe7551cf700 1 -- 172.21.15.32:0/1606391226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750106940 msgr2=0x7fe750106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.363+0000 7fe7551cf700 1 --2- 172.21.15.32:0/1606391226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750106940 0x7fe750106d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe740009990 tx=0x7fe740009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.364+0000 7fe7551cf700 1 -- 172.21.15.32:0/1606391226 shutdown_connections 2023-08-11T15:45:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.364+0000 7fe7551cf700 1 --2- 172.21.15.32:0/1606391226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750106940 0x7fe750106d20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.364+0000 7fe7551cf700 1 --2- 172.21.15.32:0/1606391226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe750101300 0x7fe75010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.364+0000 7fe7551cf700 1 --2- 172.21.15.32:0/1606391226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe750100940 0x7fe750100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.364+0000 7fe7551cf700 1 -- 172.21.15.32:0/1606391226 >> 172.21.15.32:0/1606391226 conn(0x7fe7500fc410 msgr2=0x7fe7500fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.364+0000 7fe7551cf700 1 -- 172.21.15.32:0/1606391226 shutdown_connections 2023-08-11T15:45:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.364+0000 7fe7551cf700 1 -- 172.21.15.32:0/1606391226 wait complete. 2023-08-11T15:45:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 Processor -- start 2023-08-11T15:45:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 -- start start 2023-08-11T15:45:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750100940 0x7fe75019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe750101300 0x7fe75019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:38.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe750106940 0x7fe750198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:38.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750199260 con 0x7fe750100940 2023-08-11T15:45:38.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501993d0 con 0x7fe750101300 2023-08-11T15:45:38.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.365+0000 7fe7551cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750199590 con 0x7fe750106940 2023-08-11T15:45:38.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750100940 0x7fe75019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750100940 0x7fe75019eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46180/0 (socket says 172.21.15.32:46180) 2023-08-11T15:45:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 -- 172.21.15.32:0/771746585 learned_addr learned my addr 172.21.15.32:0/771746585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe74ed9d700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe750101300 0x7fe75019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe74f59e700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe750106940 0x7fe750198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 -- 172.21.15.32:0/771746585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe750106940 msgr2=0x7fe750198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe750106940 0x7fe750198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 -- 172.21.15.32:0/771746585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe750101300 msgr2=0x7fe75019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe750101300 0x7fe75019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 -- 172.21.15.32:0/771746585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400095f0 con 0x7fe750100940 2023-08-11T15:45:38.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.366+0000 7fe746d9d700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750100940 0x7fe75019eac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe740005910 tx=0x7fe740005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.367+0000 7fe74cd99700 1 -- 172.21.15.32:0/771746585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74001d070 con 0x7fe750100940 2023-08-11T15:45:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.367+0000 7fe74cd99700 1 -- 172.21.15.32:0/771746585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe740022470 con 0x7fe750100940 2023-08-11T15:45:38.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.367+0000 7fe74cd99700 1 -- 172.21.15.32:0/771746585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740017650 con 0x7fe750100940 2023-08-11T15:45:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.367+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe750199810 con 0x7fe750100940 2023-08-11T15:45:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.367+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7501a5a40 con 0x7fe750100940 2023-08-11T15:45:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.368+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7501999d0 con 0x7fe750100940 2023-08-11T15:45:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.370+0000 7fe74cd99700 1 -- 172.21.15.32:0/771746585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fe7400225e0 con 0x7fe750100940 2023-08-11T15:45:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.372+0000 7fe74cd99700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe73c09ffe0 0x7fe73c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.373+0000 7fe74cd99700 1 -- 172.21.15.32:0/771746585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe7400cd060 con 0x7fe750100940 2023-08-11T15:45:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.373+0000 7fe74ed9d700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe73c09ffe0 0x7fe73c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.373+0000 7fe74ed9d700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe73c09ffe0 0x7fe73c0a24a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe75019a2e0 tx=0x7fe738009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.375+0000 7fe74cd99700 1 -- 172.21.15.32:0/771746585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe7400969b0 con 0x7fe750100940 2023-08-11T15:45:38.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:38 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:45:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:38 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:45:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:38 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:38.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:38 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:45:38.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:38 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:45:38.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:38 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' 2023-08-11T15:45:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.617+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe7501999a0 con 0x7fe750100940 2023-08-11T15:45:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.617+0000 7fe74cd99700 1 -- 172.21.15.32:0/771746585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe740099fd0 con 0x7fe750100940 2023-08-11T15:45:38.621 INFO:teuthology.orchestra.run.smithi032.stdout:77309411398 2023-08-11T15:45:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe73c09ffe0 msgr2=0x7fe73c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe73c09ffe0 0x7fe73c0a24a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe75019a2e0 tx=0x7fe738009260 comp rx=0 tx=0).stop 2023-08-11T15:45:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750100940 msgr2=0x7fe75019eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750100940 0x7fe75019eac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe740005910 tx=0x7fe740005720 comp rx=0 tx=0).stop 2023-08-11T15:45:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 shutdown_connections 2023-08-11T15:45:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe73c09ffe0 0x7fe73c0a24a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe750100940 0x7fe75019eac0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe750106940 0x7fe750198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 --2- 172.21.15.32:0/771746585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe750101300 0x7fe75019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 >> 172.21.15.32:0/771746585 conn(0x7fe7500fc410 msgr2=0x7fe75010bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 shutdown_connections 2023-08-11T15:45:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:38.620+0000 7fe7551cf700 1 -- 172.21.15.32:0/771746585 wait complete. 2023-08-11T15:45:39.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299437 for osd.7 2023-08-11T15:45:39.023 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:39.185 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:39 smithi032 ceph-mon[97103]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:39.186 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:39 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/771746585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-11T15:45:39.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:39 smithi134 ceph-mon[130447]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:39.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:39 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/771746585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-11T15:45:39.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:39 smithi192 ceph-mon[130806]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:39.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:39 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/771746585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-11T15:45:39.858 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411398 for osd.2 2023-08-11T15:45:39.859 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:39.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2079554146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c106940 msgr2=0x7f516c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 --2- 172.21.15.32:0/2079554146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c106940 0x7f516c106d20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5168009990 tx=0x7f5168009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2079554146 shutdown_connections 2023-08-11T15:45:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 --2- 172.21.15.32:0/2079554146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c101300 0x7f516c10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 --2- 172.21.15.32:0/2079554146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c106940 0x7f516c106d20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:39.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 --2- 172.21.15.32:0/2079554146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c100940 0x7f516c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:39.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2079554146 >> 172.21.15.32:0/2079554146 conn(0x7f516c0fc410 msgr2=0x7f516c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:39.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2079554146 shutdown_connections 2023-08-11T15:45:39.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2079554146 wait complete. 2023-08-11T15:45:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 Processor -- start 2023-08-11T15:45:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- start start 2023-08-11T15:45:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c100940 0x7f516c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c101300 0x7f516c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c106940 0x7f516c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c199100 con 0x7f516c106940 2023-08-11T15:45:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c199270 con 0x7f516c101300 2023-08-11T15:45:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c199430 con 0x7f516c100940 2023-08-11T15:45:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5171730700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c100940 0x7f516c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5171f31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c106940 0x7f516c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5171730700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c100940 0x7f516c19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46946/0 (socket says 172.21.15.32:46946) 2023-08-11T15:45:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5171730700 1 -- 172.21.15.32:0/2622313580 learned_addr learned my addr 172.21.15.32:0/2622313580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5170f2f700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c101300 0x7f516c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:39.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5170f2f700 1 -- 172.21.15.32:0/2622313580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c100940 msgr2=0x7f516c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:39.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5170f2f700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c100940 0x7f516c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:39.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5170f2f700 1 -- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c106940 msgr2=0x7f516c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5170f2f700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c106940 0x7f516c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5170f2f700 1 -- 172.21.15.32:0/2622313580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680095f0 con 0x7f516c101300 2023-08-11T15:45:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5171f31700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c106940 0x7f516c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5170f2f700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c101300 0x7f516c19ef50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f515c00e9f0 tx=0x7f515c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f51627fc700 1 -- 172.21.15.32:0/2622313580 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00eee0 con 0x7f516c101300 2023-08-11T15:45:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f51627fc700 1 -- 172.21.15.32:0/2622313580 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515c00cbd0 con 0x7f516c101300 2023-08-11T15:45:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c199710 con 0x7f516c101300 2023-08-11T15:45:39.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f51627fc700 1 -- 172.21.15.32:0/2622313580 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c018680 con 0x7f516c101300 2023-08-11T15:45:39.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c1a5a00 con 0x7f516c101300 2023-08-11T15:45:39.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.888+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c04e620 con 0x7f516c101300 2023-08-11T15:45:39.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.896+0000 7f51627fc700 1 -- 172.21.15.32:0/2622313580 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f515c00cd40 con 0x7f516c101300 2023-08-11T15:45:39.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.896+0000 7f51627fc700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f51580a0060 0x7f51580a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.896+0000 7f51627fc700 1 -- 172.21.15.32:0/2622313580 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f515c014070 con 0x7f516c101300 2023-08-11T15:45:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.896+0000 7f5171730700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f51580a0060 0x7f51580a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.896+0000 7f5171730700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f51580a0060 0x7f51580a2520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5168006010 tx=0x7f516800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:39.896+0000 7f51627fc700 1 -- 172.21.15.32:0/2622313580 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f515c095280 con 0x7f516c101300 2023-08-11T15:45:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.111+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f516c0008d0 con 0x7f516c101300 2023-08-11T15:45:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.112+0000 7f51627fc700 1 -- 172.21.15.32:0/2622313580 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f515c0988a0 con 0x7f516c101300 2023-08-11T15:45:40.115 INFO:teuthology.orchestra.run.smithi032.stdout:146028888124 2023-08-11T15:45:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.114+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f51580a0060 msgr2=0x7f51580a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.114+0000 7f5173994700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f51580a0060 0x7f51580a2520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5168006010 tx=0x7f516800b260 comp rx=0 tx=0).stop 2023-08-11T15:45:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.114+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c101300 msgr2=0x7f516c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.114+0000 7f5173994700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c101300 0x7f516c19ef50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f515c00e9f0 tx=0x7f515c00ed00 comp rx=0 tx=0).stop 2023-08-11T15:45:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 shutdown_connections 2023-08-11T15:45:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f51580a0060 0x7f51580a2520 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c106940 0x7f516c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f516c100940 0x7f516c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 --2- 172.21.15.32:0/2622313580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f516c101300 0x7f516c19ef50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 >> 172.21.15.32:0/2622313580 conn(0x7f516c0fc410 msgr2=0x7f516c10bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 shutdown_connections 2023-08-11T15:45:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.115+0000 7f5173994700 1 -- 172.21.15.32:0/2622313580 wait complete. 2023-08-11T15:45:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 -- 172.21.15.32:0/3699816898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80106940 msgr2=0x7f2a80106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:40.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 --2- 172.21.15.32:0/3699816898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80106940 0x7f2a80106d20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009990 tx=0x7f2a70009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 -- 172.21.15.32:0/3699816898 shutdown_connections 2023-08-11T15:45:40.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 --2- 172.21.15.32:0/3699816898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80106940 0x7f2a80106d20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 --2- 172.21.15.32:0/3699816898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a80101300 0x7f2a8010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 --2- 172.21.15.32:0/3699816898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80100940 0x7f2a80100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 -- 172.21.15.32:0/3699816898 >> 172.21.15.32:0/3699816898 conn(0x7f2a800fc410 msgr2=0x7f2a800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 -- 172.21.15.32:0/3699816898 shutdown_connections 2023-08-11T15:45:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.279+0000 7f2a856a6700 1 -- 172.21.15.32:0/3699816898 wait complete. 2023-08-11T15:45:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.280+0000 7f2a856a6700 1 Processor -- start 2023-08-11T15:45:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.280+0000 7f2a856a6700 1 -- start start 2023-08-11T15:45:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a856a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100940 0x7f2a8019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a856a6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101300 0x7f2a8019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a856a6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a80106940 0x7f2a80198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a856a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80199190 con 0x7f2a80100940 2023-08-11T15:45:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a856a6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80199300 con 0x7f2a80101300 2023-08-11T15:45:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a856a6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801994c0 con 0x7f2a80106940 2023-08-11T15:45:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a7effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100940 0x7f2a8019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a7effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100940 0x7f2a8019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46212/0 (socket says 172.21.15.32:46212) 2023-08-11T15:45:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a7effd700 1 -- 172.21.15.32:0/2547416474 learned_addr learned my addr 172.21.15.32:0/2547416474 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a7e7fc700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101300 0x7f2a8019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.281+0000 7f2a7f7fe700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a80106940 0x7f2a80198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a7e7fc700 1 -- 172.21.15.32:0/2547416474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a80106940 msgr2=0x7f2a80198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a7e7fc700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a80106940 0x7f2a80198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a7e7fc700 1 -- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100940 msgr2=0x7f2a8019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a7e7fc700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100940 0x7f2a8019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a7e7fc700 1 -- 172.21.15.32:0/2547416474 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700095f0 con 0x7f2a80101300 2023-08-11T15:45:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a7e7fc700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101300 0x7f2a8019ef90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e9f0 tx=0x7f2a7400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a67fff700 1 -- 172.21.15.32:0/2547416474 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400c9c0 con 0x7f2a80101300 2023-08-11T15:45:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a67fff700 1 -- 172.21.15.32:0/2547416474 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a74010430 con 0x7f2a80101300 2023-08-11T15:45:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a801997a0 con 0x7f2a80101300 2023-08-11T15:45:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.282+0000 7f2a67fff700 1 -- 172.21.15.32:0/2547416474 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74018680 con 0x7f2a80101300 2023-08-11T15:45:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.283+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a801a5a40 con 0x7f2a80101300 2023-08-11T15:45:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.284+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8004e620 con 0x7f2a80101300 2023-08-11T15:45:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.285+0000 7f2a67fff700 1 -- 172.21.15.32:0/2547416474 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f2a74010860 con 0x7f2a80101300 2023-08-11T15:45:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.288+0000 7f2a67fff700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a6809ffe0 0x7f2a680a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.288+0000 7f2a67fff700 1 -- 172.21.15.32:0/2547416474 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2a74014070 con 0x7f2a80101300 2023-08-11T15:45:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.288+0000 7f2a7effd700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a6809ffe0 0x7f2a680a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.289+0000 7f2a7effd700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a6809ffe0 0x7f2a680a24a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a7000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:40.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.289+0000 7f2a67fff700 1 -- 172.21.15.32:0/2547416474 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2a74095160 con 0x7f2a80101300 2023-08-11T15:45:40.416 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:40 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2622313580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-11T15:45:40.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:40 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2622313580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-11T15:45:40.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.535+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2a80199fe0 con 0x7f2a80101300 2023-08-11T15:45:40.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.536+0000 7f2a67fff700 1 -- 172.21.15.32:0/2547416474 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a74098780 con 0x7f2a80101300 2023-08-11T15:45:40.539 INFO:teuthology.orchestra.run.smithi032.stdout:201863462963 2023-08-11T15:45:40.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a6809ffe0 msgr2=0x7f2a680a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a6809ffe0 0x7f2a680a24a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a7000b260 comp rx=0 tx=0).stop 2023-08-11T15:45:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101300 msgr2=0x7f2a8019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:40.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101300 0x7f2a8019ef90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e9f0 tx=0x7f2a7400ed00 comp rx=0 tx=0).stop 2023-08-11T15:45:40.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 shutdown_connections 2023-08-11T15:45:40.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f2a6809ffe0 0x7f2a680a24a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a80100940 0x7f2a8019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a80106940 0x7f2a80198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 --2- 172.21.15.32:0/2547416474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101300 0x7f2a8019ef90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:40.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 >> 172.21.15.32:0/2547416474 conn(0x7f2a800fc410 msgr2=0x7f2a800fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:40.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 shutdown_connections 2023-08-11T15:45:40.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:40.539+0000 7f2a856a6700 1 -- 172.21.15.32:0/2547416474 wait complete. 2023-08-11T15:45:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:40 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2622313580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-11T15:45:40.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888124 for osd.4 2023-08-11T15:45:40.775 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:41.330 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462960 got 201863462963 for osd.6 2023-08-11T15:45:41.331 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:41.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:41 smithi134 ceph-mon[130447]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:41 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2547416474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-11T15:45:41.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 -- 172.21.15.32:0/1985724942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 msgr2=0x7fa1e010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:41.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 --2- 172.21.15.32:0/1985724942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 0x7fa1e010d810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b0a0 tx=0x7fa1d800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 -- 172.21.15.32:0/1985724942 shutdown_connections 2023-08-11T15:45:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 --2- 172.21.15.32:0/1985724942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 0x7fa1e010d810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 --2- 172.21.15.32:0/1985724942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1e0106940 0x7fa1e0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 --2- 172.21.15.32:0/1985724942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 0x7fa1e0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 -- 172.21.15.32:0/1985724942 >> 172.21.15.32:0/1985724942 conn(0x7fa1e00fc410 msgr2=0x7fa1e00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 -- 172.21.15.32:0/1985724942 shutdown_connections 2023-08-11T15:45:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.535+0000 7fa1e722a700 1 -- 172.21.15.32:0/1985724942 wait complete. 2023-08-11T15:45:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.536+0000 7fa1e722a700 1 Processor -- start 2023-08-11T15:45:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.536+0000 7fa1e722a700 1 -- start start 2023-08-11T15:45:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e722a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 0x7fa1e0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e722a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 0x7fa1e0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e722a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1e0106940 0x7fa1e00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e722a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e01a4fb0 con 0x7fa1e0100940 2023-08-11T15:45:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e722a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e01a5120 con 0x7fa1e0101300 2023-08-11T15:45:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e722a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e01a52e0 con 0x7fa1e0106940 2023-08-11T15:45:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 0x7fa1e0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 0x7fa1e0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46238/0 (socket says 172.21.15.32:46238) 2023-08-11T15:45:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 -- 172.21.15.32:0/2698407343 learned_addr learned my addr 172.21.15.32:0/2698407343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e57c7700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1e0106940 0x7fa1e00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1dffff700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 0x7fa1e0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 -- 172.21.15.32:0/2698407343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1e0106940 msgr2=0x7fa1e00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1e0106940 0x7fa1e00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 -- 172.21.15.32:0/2698407343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 msgr2=0x7fa1e0073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 0x7fa1e0073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.537+0000 7fa1e4fc6700 1 -- 172.21.15.32:0/2698407343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8009d00 con 0x7fa1e0100940 2023-08-11T15:45:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.538+0000 7fa1e4fc6700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 0x7fa1e0072c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9c0 tx=0x7fa1d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.538+0000 7fa1ddffb700 1 -- 172.21.15.32:0/2698407343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40097b0 con 0x7fa1e0100940 2023-08-11T15:45:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.538+0000 7fa1ddffb700 1 -- 172.21.15.32:0/2698407343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d400ee60 con 0x7fa1e0100940 2023-08-11T15:45:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.538+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e01a55c0 con 0x7fa1e0100940 2023-08-11T15:45:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.538+0000 7fa1ddffb700 1 -- 172.21.15.32:0/2698407343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40054f0 con 0x7fa1e0100940 2023-08-11T15:45:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.538+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e01a5ac0 con 0x7fa1e0100940 2023-08-11T15:45:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.539+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e004e620 con 0x7fa1e0100940 2023-08-11T15:45:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.541+0000 7fa1ddffb700 1 -- 172.21.15.32:0/2698407343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa1d4004750 con 0x7fa1e0100940 2023-08-11T15:45:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.542+0000 7fa1ddffb700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa1d00a0060 0x7fa1d00a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.543+0000 7fa1ddffb700 1 -- 172.21.15.32:0/2698407343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1d4014070 con 0x7fa1e0100940 2023-08-11T15:45:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.543+0000 7fa1dffff700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa1d00a0060 0x7fa1d00a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.543+0000 7fa1dffff700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa1d00a0060 0x7fa1d00a2520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0074410 tx=0x7fa1cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.547+0000 7fa1ddffb700 1 -- 172.21.15.32:0/2698407343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa1d4095110 con 0x7fa1e0100940 2023-08-11T15:45:41.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:41 smithi032 ceph-mon[97103]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:41.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:41 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2547416474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-11T15:45:41.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:41 smithi192 ceph-mon[130806]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:41.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:41 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2547416474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-11T15:45:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.760+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fa1e0110a80 con 0x7fa1e0100940 2023-08-11T15:45:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.760+0000 7fa1ddffb700 1 -- 172.21.15.32:0/2698407343 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa1d4098730 con 0x7fa1e0100940 2023-08-11T15:45:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:253403070507 2023-08-11T15:45:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa1d00a0060 msgr2=0x7fa1d00a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa1d00a0060 0x7fa1d00a2520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0074410 tx=0x7fa1cc009260 comp rx=0 tx=0).stop 2023-08-11T15:45:41.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 msgr2=0x7fa1e0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 0x7fa1e0072c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9c0 tx=0x7fa1d400ecd0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 shutdown_connections 2023-08-11T15:45:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa1d00a0060 0x7fa1d00a2520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e0100940 0x7fa1e0072c20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1e0106940 0x7fa1e00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 --2- 172.21.15.32:0/2698407343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1e0101300 0x7fa1e0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 >> 172.21.15.32:0/2698407343 conn(0x7fa1e00fc410 msgr2=0x7fa1e00fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 shutdown_connections 2023-08-11T15:45:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:41.763+0000 7fa1e722a700 1 -- 172.21.15.32:0/2698407343 wait complete. 2023-08-11T15:45:42.292 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070507 for osd.8 2023-08-11T15:45:42.292 DEBUG:teuthology.parallel:result is None 2023-08-11T15:45:42.293 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-08-11T15:45:42.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph pg dump --format=json 2023-08-11T15:45:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:42 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2698407343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-11T15:45:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:42 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2698407343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-11T15:45:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:42 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2698407343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-11T15:45:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:43.440 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:43 smithi032 ceph-mon[97103]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:43.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:43 smithi134 ceph-mon[130447]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:43.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:43 smithi192 ceph-mon[130806]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 -- 172.21.15.32:0/209289587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8100940 msgr2=0x7f43a8100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 --2- 172.21.15.32:0/209289587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8100940 0x7f43a8100dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4394009990 tx=0x7f4394009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 -- 172.21.15.32:0/209289587 shutdown_connections 2023-08-11T15:45:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 --2- 172.21.15.32:0/209289587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8101300 0x7f43a810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 --2- 172.21.15.32:0/209289587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a8106940 0x7f43a8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 --2- 172.21.15.32:0/209289587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8100940 0x7f43a8100dc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 -- 172.21.15.32:0/209289587 >> 172.21.15.32:0/209289587 conn(0x7f43a80fc410 msgr2=0x7f43a80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 -- 172.21.15.32:0/209289587 shutdown_connections 2023-08-11T15:45:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.559+0000 7f43aee1c700 1 -- 172.21.15.32:0/209289587 wait complete. 2023-08-11T15:45:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.560+0000 7f43aee1c700 1 Processor -- start 2023-08-11T15:45:44.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.560+0000 7f43aee1c700 1 -- start start 2023-08-11T15:45:44.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.560+0000 7f43aee1c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8100940 0x7f43a819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:44.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.560+0000 7f43aee1c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a8101300 0x7f43a819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:44.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43aee1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8106940 0x7f43a81968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:44.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43aee1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8196f10 con 0x7f43a8106940 2023-08-11T15:45:44.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43aee1c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8197080 con 0x7f43a8100940 2023-08-11T15:45:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43aee1c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8197240 con 0x7f43a8101300 2023-08-11T15:45:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8100940 0x7f43a819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43a7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a8101300 0x7f43a819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8100940 0x7f43a819c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:33214/0 (socket says 172.21.15.32:33214) 2023-08-11T15:45:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43ad3b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8106940 0x7f43a81968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 -- 172.21.15.32:0/657108083 learned_addr learned my addr 172.21.15.32:0/657108083 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:44.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 -- 172.21.15.32:0/657108083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a8101300 msgr2=0x7f43a819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a8101300 0x7f43a819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 -- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8106940 msgr2=0x7f43a81968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8106940 0x7f43a81968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43acbb8700 1 -- 172.21.15.32:0/657108083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43940095f0 con 0x7f43a8100940 2023-08-11T15:45:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.561+0000 7f43ad3b9700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8106940 0x7f43a81968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:44.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.562+0000 7f43acbb8700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8100940 0x7f43a819c820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f439c00d950 tx=0x7f439c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.562+0000 7f43a5ffb700 1 -- 172.21.15.32:0/657108083 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00f890 con 0x7f43a8100940 2023-08-11T15:45:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.562+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a8197520 con 0x7f43a8100940 2023-08-11T15:45:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.562+0000 7f43a5ffb700 1 -- 172.21.15.32:0/657108083 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f439c010460 con 0x7f43a8100940 2023-08-11T15:45:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.562+0000 7f43a5ffb700 1 -- 172.21.15.32:0/657108083 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00e610 con 0x7f43a8100940 2023-08-11T15:45:44.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.562+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a8072d20 con 0x7f43a8100940 2023-08-11T15:45:44.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.563+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a804e620 con 0x7f43a8100940 2023-08-11T15:45:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.565+0000 7f43a5ffb700 1 -- 172.21.15.32:0/657108083 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f439c0108d0 con 0x7f43a8100940 2023-08-11T15:45:44.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.566+0000 7f43a5ffb700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f439809ffe0 0x7f43980a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.567+0000 7f43a5ffb700 1 -- 172.21.15.32:0/657108083 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f439c0cb480 con 0x7f43a8100940 2023-08-11T15:45:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.567+0000 7f43a7fff700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f439809ffe0 0x7f43980a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.567+0000 7f43a7fff700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f439809ffe0 0x7f43980a24a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43a8106070 tx=0x7f4394005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.569+0000 7f43a5ffb700 1 -- 172.21.15.32:0/657108083 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f439c094d40 con 0x7f43a8100940 2023-08-11T15:45:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.774+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43a80008d0 con 0x7f439809ffe0 2023-08-11T15:45:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.788+0000 7f43a5ffb700 1 -- 172.21.15.32:0/657108083 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426368 (secure 0 0 0) 0x7f43a80008d0 con 0x7f439809ffe0 2023-08-11T15:45:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f439809ffe0 msgr2=0x7f43980a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f439809ffe0 0x7f43980a24a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43a8106070 tx=0x7f4394005fd0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8100940 msgr2=0x7f43a819c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8100940 0x7f43a819c820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f439c00d950 tx=0x7f439c00dc60 comp rx=0 tx=0).stop 2023-08-11T15:45:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 shutdown_connections 2023-08-11T15:45:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f439809ffe0 0x7f43980a24a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a8106940 0x7f43a81968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8100940 0x7f43a819c820 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 --2- 172.21.15.32:0/657108083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a8101300 0x7f43a819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 >> 172.21.15.32:0/657108083 conn(0x7f43a80fc410 msgr2=0x7f43a80fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 shutdown_connections 2023-08-11T15:45:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:44.794+0000 7f43aee1c700 1 -- 172.21.15.32:0/657108083 wait complete. 2023-08-11T15:45:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-08-11T15:45:45.334 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2023-08-11T15:45:44.090165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652882,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":253,"num_read_kb":240,"num_write":256,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337936,"kb_used_data":13688,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603888,"statfs":{"total":1151940427776,"available":1151594381312,"internally_reserved":0,"allocated":14016512,"data_stored":7824167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003398"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192439+0000","last_change":"2023-08-11T15:44:19.862927+0000","last_active":"2023-08-11T15:44:55.192439+0000","last_peered":"2023-08-11T15:44:55.192439+0000","last_clean":"2023-08-11T15:44:55.192439+0000","last_became_active":"2023-08-11T15:44:19.862626+0000","last_became_peered":"2023-08-11T15:44:19.862626+0000","last_unstale":"2023-08-11T15:44:55.192439+0000","last_undegraded":"2023-08-11T15:44:55.192439+0000","last_fullsized":"2023-08-11T15:44:55.192439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:39:26.212373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190047+0000","last_change":"2023-08-11T15:44:19.852455+0000","last_active":"2023-08-11T15:44:55.190047+0000","last_peered":"2023-08-11T15:44:55.190047+0000","last_clean":"2023-08-11T15:44:55.190047+0000","last_became_active":"2023-08-11T15:44:19.851897+0000","last_became_peered":"2023-08-11T15:44:19.851897+0000","last_unstale":"2023-08-11T15:44:55.190047+0000","last_undegraded":"2023-08-11T15:44:55.190047+0000","last_fullsized":"2023-08-11T15:44:55.190047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:23:56.473990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189740+0000","last_change":"2023-08-11T15:44:19.855232+0000","last_active":"2023-08-11T15:44:55.189740+0000","last_peered":"2023-08-11T15:44:55.189740+0000","last_clean":"2023-08-11T15:44:55.189740+0000","last_became_active":"2023-08-11T15:44:19.854940+0000","last_became_peered":"2023-08-11T15:44:19.854940+0000","last_unstale":"2023-08-11T15:44:55.189740+0000","last_undegraded":"2023-08-11T15:44:55.189740+0000","last_fullsized":"2023-08-11T15:44:55.189740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:58:15.175180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198360+0000","last_change":"2023-08-11T15:44:19.856137+0000","last_active":"2023-08-11T15:44:55.198360+0000","last_peered":"2023-08-11T15:44:55.198360+0000","last_clean":"2023-08-11T15:44:55.198360+0000","last_became_active":"2023-08-11T15:44:19.855852+0000","last_became_peered":"2023-08-11T15:44:19.855852+0000","last_unstale":"2023-08-11T15:44:55.198360+0000","last_undegraded":"2023-08-11T15:44:55.198360+0000","last_fullsized":"2023-08-11T15:44:55.198360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:50:01.697765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194593+0000","last_change":"2023-08-11T15:44:19.862618+0000","last_active":"2023-08-11T15:44:55.194593+0000","last_peered":"2023-08-11T15:44:55.194593+0000","last_clean":"2023-08-11T15:44:55.194593+0000","last_became_active":"2023-08-11T15:44:19.862265+0000","last_became_peered":"2023-08-11T15:44:19.862265+0000","last_unstale":"2023-08-11T15:44:55.194593+0000","last_undegraded":"2023-08-11T15:44:55.194593+0000","last_fullsized":"2023-08-11T15:44:55.194593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:40.449974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192805+0000","last_change":"2023-08-11T15:44:19.857801+0000","last_active":"2023-08-11T15:44:55.192805+0000","last_peered":"2023-08-11T15:44:55.192805+0000","last_clean":"2023-08-11T15:44:55.192805+0000","last_became_active":"2023-08-11T15:44:19.857388+0000","last_became_peered":"2023-08-11T15:44:19.857388+0000","last_unstale":"2023-08-11T15:44:55.192805+0000","last_undegraded":"2023-08-11T15:44:55.192805+0000","last_fullsized":"2023-08-11T15:44:55.192805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:09:13.938322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194812+0000","last_change":"2023-08-11T15:44:19.859334+0000","last_active":"2023-08-11T15:44:55.194812+0000","last_peered":"2023-08-11T15:44:55.194812+0000","last_clean":"2023-08-11T15:44:55.194812+0000","last_became_active":"2023-08-11T15:44:19.859073+0000","last_became_peered":"2023-08-11T15:44:19.859073+0000","last_unstale":"2023-08-11T15:44:55.194812+0000","last_undegraded":"2023-08-11T15:44:55.194812+0000","last_fullsized":"2023-08-11T15:44:55.194812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:40:27.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194562+0000","last_change":"2023-08-11T15:44:19.853064+0000","last_active":"2023-08-11T15:44:55.194562+0000","last_peered":"2023-08-11T15:44:55.194562+0000","last_clean":"2023-08-11T15:44:55.194562+0000","last_became_active":"2023-08-11T15:44:19.852736+0000","last_became_peered":"2023-08-11T15:44:19.852736+0000","last_unstale":"2023-08-11T15:44:55.194562+0000","last_undegraded":"2023-08-11T15:44:55.194562+0000","last_fullsized":"2023-08-11T15:44:55.194562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:57:13.446578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191401+0000","last_change":"2023-08-11T15:44:19.861912+0000","last_active":"2023-08-11T15:44:55.191401+0000","last_peered":"2023-08-11T15:44:55.191401+0000","last_clean":"2023-08-11T15:44:55.191401+0000","last_became_active":"2023-08-11T15:44:19.861374+0000","last_became_peered":"2023-08-11T15:44:19.861374+0000","last_unstale":"2023-08-11T15:44:55.191401+0000","last_undegraded":"2023-08-11T15:44:55.191401+0000","last_fullsized":"2023-08-11T15:44:55.191401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:43:22.890968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193523+0000","last_change":"2023-08-11T15:44:19.858862+0000","last_active":"2023-08-11T15:44:55.193523+0000","last_peered":"2023-08-11T15:44:55.193523+0000","last_clean":"2023-08-11T15:44:55.193523+0000","last_became_active":"2023-08-11T15:44:19.858512+0000","last_became_peered":"2023-08-11T15:44:19.858512+0000","last_unstale":"2023-08-11T15:44:55.193523+0000","last_undegraded":"2023-08-11T15:44:55.193523+0000","last_fullsized":"2023-08-11T15:44:55.193523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:40:43.630548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195034+0000","last_change":"2023-08-11T15:44:19.864063+0000","last_active":"2023-08-11T15:44:55.195034+0000","last_peered":"2023-08-11T15:44:55.195034+0000","last_clean":"2023-08-11T15:44:55.195034+0000","last_became_active":"2023-08-11T15:44:19.863081+0000","last_became_peered":"2023-08-11T15:44:19.863081+0000","last_unstale":"2023-08-11T15:44:55.195034+0000","last_undegraded":"2023-08-11T15:44:55.195034+0000","last_fullsized":"2023-08-11T15:44:55.195034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:58:18.278385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198186+0000","last_change":"2023-08-11T15:44:19.855347+0000","last_active":"2023-08-11T15:44:55.198186+0000","last_peered":"2023-08-11T15:44:55.198186+0000","last_clean":"2023-08-11T15:44:55.198186+0000","last_became_active":"2023-08-11T15:44:19.855025+0000","last_became_peered":"2023-08-11T15:44:19.855025+0000","last_unstale":"2023-08-11T15:44:55.198186+0000","last_undegraded":"2023-08-11T15:44:55.198186+0000","last_fullsized":"2023-08-11T15:44:55.198186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:44:49.739947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195827+0000","last_change":"2023-08-11T15:44:19.862153+0000","last_active":"2023-08-11T15:44:55.195827+0000","last_peered":"2023-08-11T15:44:55.195827+0000","last_clean":"2023-08-11T15:44:55.195827+0000","last_became_active":"2023-08-11T15:44:19.861830+0000","last_became_peered":"2023-08-11T15:44:19.861830+0000","last_unstale":"2023-08-11T15:44:55.195827+0000","last_undegraded":"2023-08-11T15:44:55.195827+0000","last_fullsized":"2023-08-11T15:44:55.195827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:16:01.026944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189564+0000","last_change":"2023-08-11T15:44:19.857184+0000","last_active":"2023-08-11T15:44:55.189564+0000","last_peered":"2023-08-11T15:44:55.189564+0000","last_clean":"2023-08-11T15:44:55.189564+0000","last_became_active":"2023-08-11T15:44:19.856898+0000","last_became_peered":"2023-08-11T15:44:19.856898+0000","last_unstale":"2023-08-11T15:44:55.189564+0000","last_undegraded":"2023-08-11T15:44:55.189564+0000","last_fullsized":"2023-08-11T15:44:55.189564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:35:04.984315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189495+0000","last_change":"2023-08-11T15:44:19.852808+0000","last_active":"2023-08-11T15:44:55.189495+0000","last_peered":"2023-08-11T15:44:55.189495+0000","last_clean":"2023-08-11T15:44:55.189495+0000","last_became_active":"2023-08-11T15:44:19.852477+0000","last_became_peered":"2023-08-11T15:44:19.852477+0000","last_unstale":"2023-08-11T15:44:55.189495+0000","last_undegraded":"2023-08-11T15:44:55.189495+0000","last_fullsized":"2023-08-11T15:44:55.189495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:15.737333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190450+0000","last_change":"2023-08-11T15:44:19.856273+0000","last_active":"2023-08-11T15:44:55.190450+0000","last_peered":"2023-08-11T15:44:55.190450+0000","last_clean":"2023-08-11T15:44:55.190450+0000","last_became_active":"2023-08-11T15:44:19.854579+0000","last_became_peered":"2023-08-11T15:44:19.854579+0000","last_unstale":"2023-08-11T15:44:55.190450+0000","last_undegraded":"2023-08-11T15:44:55.190450+0000","last_fullsized":"2023-08-11T15:44:55.190450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:02:07.058213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192239+0000","last_change":"2023-08-11T15:44:19.866820+0000","last_active":"2023-08-11T15:44:55.192239+0000","last_peered":"2023-08-11T15:44:55.192239+0000","last_clean":"2023-08-11T15:44:55.192239+0000","last_became_active":"2023-08-11T15:44:19.866507+0000","last_became_peered":"2023-08-11T15:44:19.866507+0000","last_unstale":"2023-08-11T15:44:55.192239+0000","last_undegraded":"2023-08-11T15:44:55.192239+0000","last_fullsized":"2023-08-11T15:44:55.192239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:41:54.967368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193782+0000","last_change":"2023-08-11T15:44:19.865821+0000","last_active":"2023-08-11T15:44:55.193782+0000","last_peered":"2023-08-11T15:44:55.193782+0000","last_clean":"2023-08-11T15:44:55.193782+0000","last_became_active":"2023-08-11T15:44:19.865510+0000","last_became_peered":"2023-08-11T15:44:19.865510+0000","last_unstale":"2023-08-11T15:44:55.193782+0000","last_undegraded":"2023-08-11T15:44:55.193782+0000","last_fullsized":"2023-08-11T15:44:55.193782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:20:32.474091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194714+0000","last_change":"2023-08-11T15:44:19.863079+0000","last_active":"2023-08-11T15:44:55.194714+0000","last_peered":"2023-08-11T15:44:55.194714+0000","last_clean":"2023-08-11T15:44:55.194714+0000","last_became_active":"2023-08-11T15:44:19.862663+0000","last_became_peered":"2023-08-11T15:44:19.862663+0000","last_unstale":"2023-08-11T15:44:55.194714+0000","last_undegraded":"2023-08-11T15:44:55.194714+0000","last_fullsized":"2023-08-11T15:44:55.194714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:41:51.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194903+0000","last_change":"2023-08-11T15:44:24.201308+0000","last_active":"2023-08-11T15:44:55.194903+0000","last_peered":"2023-08-11T15:44:55.194903+0000","last_clean":"2023-08-11T15:44:55.194903+0000","last_became_active":"2023-08-11T15:44:24.200884+0000","last_became_peered":"2023-08-11T15:44:24.200884+0000","last_unstale":"2023-08-11T15:44:55.194903+0000","last_undegraded":"2023-08-11T15:44:55.194903+0000","last_fullsized":"2023-08-11T15:44:55.194903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:59.419400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193546+0000","last_change":"2023-08-11T15:44:19.155621+0000","last_active":"2023-08-11T15:44:55.193546+0000","last_peered":"2023-08-11T15:44:55.193546+0000","last_clean":"2023-08-11T15:44:55.193546+0000","last_became_active":"2023-08-11T15:44:19.154494+0000","last_became_peered":"2023-08-11T15:44:19.154494+0000","last_unstale":"2023-08-11T15:44:55.193546+0000","last_undegraded":"2023-08-11T15:44:55.193546+0000","last_fullsized":"2023-08-11T15:44:55.193546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:09.243769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196857+0000","last_change":"2023-08-11T15:44:19.853101+0000","last_active":"2023-08-11T15:44:55.196857+0000","last_peered":"2023-08-11T15:44:55.196857+0000","last_clean":"2023-08-11T15:44:55.196857+0000","last_became_active":"2023-08-11T15:44:19.852804+0000","last_became_peered":"2023-08-11T15:44:19.852804+0000","last_unstale":"2023-08-11T15:44:55.196857+0000","last_undegraded":"2023-08-11T15:44:55.196857+0000","last_fullsized":"2023-08-11T15:44:55.196857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:26:49.173800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200726+0000","last_change":"2023-08-11T15:44:55.200726+0000","last_active":"2023-08-11T15:44:55.200726+0000","last_peered":"2023-08-11T15:44:55.200726+0000","last_clean":"2023-08-11T15:44:55.200726+0000","last_became_active":"2023-08-11T15:44:55.200360+0000","last_became_peered":"2023-08-11T15:44:55.200360+0000","last_unstale":"2023-08-11T15:44:55.200726+0000","last_undegraded":"2023-08-11T15:44:55.200726+0000","last_fullsized":"2023-08-11T15:44:55.200726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:59:09.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196984+0000","last_change":"2023-08-11T15:44:19.147219+0000","last_active":"2023-08-11T15:44:55.196984+0000","last_peered":"2023-08-11T15:44:55.196984+0000","last_clean":"2023-08-11T15:44:55.196984+0000","last_became_active":"2023-08-11T15:44:19.146874+0000","last_became_peered":"2023-08-11T15:44:19.146874+0000","last_unstale":"2023-08-11T15:44:55.196984+0000","last_undegraded":"2023-08-11T15:44:55.196984+0000","last_fullsized":"2023-08-11T15:44:55.196984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:33.758701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189983+0000","last_change":"2023-08-11T15:44:19.854103+0000","last_active":"2023-08-11T15:44:55.189983+0000","last_peered":"2023-08-11T15:44:55.189983+0000","last_clean":"2023-08-11T15:44:55.189983+0000","last_became_active":"2023-08-11T15:44:19.853542+0000","last_became_peered":"2023-08-11T15:44:19.853542+0000","last_unstale":"2023-08-11T15:44:55.189983+0000","last_undegraded":"2023-08-11T15:44:55.189983+0000","last_fullsized":"2023-08-11T15:44:55.189983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:35:27.247168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200630+0000","last_change":"2023-08-11T15:44:55.200630+0000","last_active":"2023-08-11T15:44:55.200630+0000","last_peered":"2023-08-11T15:44:55.200630+0000","last_clean":"2023-08-11T15:44:55.200630+0000","last_became_active":"2023-08-11T15:44:55.200271+0000","last_became_peered":"2023-08-11T15:44:55.200271+0000","last_unstale":"2023-08-11T15:44:55.200630+0000","last_undegraded":"2023-08-11T15:44:55.200630+0000","last_fullsized":"2023-08-11T15:44:55.200630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:50:50.162322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195272+0000","last_change":"2023-08-11T15:44:19.148019+0000","last_active":"2023-08-11T15:44:55.195272+0000","last_peered":"2023-08-11T15:44:55.195272+0000","last_clean":"2023-08-11T15:44:55.195272+0000","last_became_active":"2023-08-11T15:44:19.147501+0000","last_became_peered":"2023-08-11T15:44:19.147501+0000","last_unstale":"2023-08-11T15:44:55.195272+0000","last_undegraded":"2023-08-11T15:44:55.195272+0000","last_fullsized":"2023-08-11T15:44:55.195272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:32.333344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190689+0000","last_change":"2023-08-11T15:44:19.865230+0000","last_active":"2023-08-11T15:44:55.190689+0000","last_peered":"2023-08-11T15:44:55.190689+0000","last_clean":"2023-08-11T15:44:55.190689+0000","last_became_active":"2023-08-11T15:44:19.864838+0000","last_became_peered":"2023-08-11T15:44:19.864838+0000","last_unstale":"2023-08-11T15:44:55.190689+0000","last_undegraded":"2023-08-11T15:44:55.190689+0000","last_fullsized":"2023-08-11T15:44:55.190689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:10.222090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198211+0000","last_change":"2023-08-11T15:44:24.192282+0000","last_active":"2023-08-11T15:44:55.198211+0000","last_peered":"2023-08-11T15:44:55.198211+0000","last_clean":"2023-08-11T15:44:55.198211+0000","last_became_active":"2023-08-11T15:44:24.191972+0000","last_became_peered":"2023-08-11T15:44:24.191972+0000","last_unstale":"2023-08-11T15:44:55.198211+0000","last_undegraded":"2023-08-11T15:44:55.198211+0000","last_fullsized":"2023-08-11T15:44:55.198211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:14:12.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195406+0000","last_change":"2023-08-11T15:44:19.147181+0000","last_active":"2023-08-11T15:44:55.195406+0000","last_peered":"2023-08-11T15:44:55.195406+0000","last_clean":"2023-08-11T15:44:55.195406+0000","last_became_active":"2023-08-11T15:44:19.146843+0000","last_became_peered":"2023-08-11T15:44:19.146843+0000","last_unstale":"2023-08-11T15:44:55.195406+0000","last_undegraded":"2023-08-11T15:44:55.195406+0000","last_fullsized":"2023-08-11T15:44:55.195406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:39:50.508765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200249+0000","last_change":"2023-08-11T15:44:55.199783+0000","last_active":"2023-08-11T15:44:55.200249+0000","last_peered":"2023-08-11T15:44:55.200249+0000","last_clean":"2023-08-11T15:44:55.200249+0000","last_became_active":"2023-08-11T15:44:55.199391+0000","last_became_peered":"2023-08-11T15:44:55.199391+0000","last_unstale":"2023-08-11T15:44:55.200249+0000","last_undegraded":"2023-08-11T15:44:55.200249+0000","last_fullsized":"2023-08-11T15:44:55.200249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:19:02.132279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194561+0000","last_change":"2023-08-11T15:44:24.192621+0000","last_active":"2023-08-11T15:44:55.194561+0000","last_peered":"2023-08-11T15:44:55.194561+0000","last_clean":"2023-08-11T15:44:55.194561+0000","last_became_active":"2023-08-11T15:44:24.192183+0000","last_became_peered":"2023-08-11T15:44:24.192183+0000","last_unstale":"2023-08-11T15:44:55.194561+0000","last_undegraded":"2023-08-11T15:44:55.194561+0000","last_fullsized":"2023-08-11T15:44:55.194561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:16.822794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190900+0000","last_change":"2023-08-11T15:44:19.149437+0000","last_active":"2023-08-11T15:44:55.190900+0000","last_peered":"2023-08-11T15:44:55.190900+0000","last_clean":"2023-08-11T15:44:55.190900+0000","last_became_active":"2023-08-11T15:44:19.149086+0000","last_became_peered":"2023-08-11T15:44:19.149086+0000","last_unstale":"2023-08-11T15:44:55.190900+0000","last_undegraded":"2023-08-11T15:44:55.190900+0000","last_fullsized":"2023-08-11T15:44:55.190900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:50.716278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195215+0000","last_change":"2023-08-11T15:44:19.865452+0000","last_active":"2023-08-11T15:44:55.195215+0000","last_peered":"2023-08-11T15:44:55.195215+0000","last_clean":"2023-08-11T15:44:55.195215+0000","last_became_active":"2023-08-11T15:44:19.865133+0000","last_became_peered":"2023-08-11T15:44:19.865133+0000","last_unstale":"2023-08-11T15:44:55.195215+0000","last_undegraded":"2023-08-11T15:44:55.195215+0000","last_fullsized":"2023-08-11T15:44:55.195215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:42:54.760249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195540+0000","last_change":"2023-08-11T15:44:24.201817+0000","last_active":"2023-08-11T15:44:55.195540+0000","last_peered":"2023-08-11T15:44:55.195540+0000","last_clean":"2023-08-11T15:44:55.195540+0000","last_became_active":"2023-08-11T15:44:24.200766+0000","last_became_peered":"2023-08-11T15:44:24.200766+0000","last_unstale":"2023-08-11T15:44:55.195540+0000","last_undegraded":"2023-08-11T15:44:55.195540+0000","last_fullsized":"2023-08-11T15:44:55.195540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:46:08.346089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192629+0000","last_change":"2023-08-11T15:44:19.151213+0000","last_active":"2023-08-11T15:44:55.192629+0000","last_peered":"2023-08-11T15:44:55.192629+0000","last_clean":"2023-08-11T15:44:55.192629+0000","last_became_active":"2023-08-11T15:44:19.150914+0000","last_became_peered":"2023-08-11T15:44:19.150914+0000","last_unstale":"2023-08-11T15:44:55.192629+0000","last_undegraded":"2023-08-11T15:44:55.192629+0000","last_fullsized":"2023-08-11T15:44:55.192629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:30.877228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191123+0000","last_change":"2023-08-11T15:44:19.858238+0000","last_active":"2023-08-11T15:44:55.191123+0000","last_peered":"2023-08-11T15:44:55.191123+0000","last_clean":"2023-08-11T15:44:55.191123+0000","last_became_active":"2023-08-11T15:44:19.857919+0000","last_became_peered":"2023-08-11T15:44:19.857919+0000","last_unstale":"2023-08-11T15:44:55.191123+0000","last_undegraded":"2023-08-11T15:44:55.191123+0000","last_fullsized":"2023-08-11T15:44:55.191123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:15:28.098691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192085+0000","last_change":"2023-08-11T15:44:24.201485+0000","last_active":"2023-08-11T15:44:55.192085+0000","last_peered":"2023-08-11T15:44:55.192085+0000","last_clean":"2023-08-11T15:44:55.192085+0000","last_became_active":"2023-08-11T15:44:24.200984+0000","last_became_peered":"2023-08-11T15:44:24.200984+0000","last_unstale":"2023-08-11T15:44:55.192085+0000","last_undegraded":"2023-08-11T15:44:55.192085+0000","last_fullsized":"2023-08-11T15:44:55.192085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:57.566307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195775+0000","last_change":"2023-08-11T15:44:19.149289+0000","last_active":"2023-08-11T15:44:55.195775+0000","last_peered":"2023-08-11T15:44:55.195775+0000","last_clean":"2023-08-11T15:44:55.195775+0000","last_became_active":"2023-08-11T15:44:19.148881+0000","last_became_peered":"2023-08-11T15:44:19.148881+0000","last_unstale":"2023-08-11T15:44:55.195775+0000","last_undegraded":"2023-08-11T15:44:55.195775+0000","last_fullsized":"2023-08-11T15:44:55.195775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:02.163702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194495+0000","last_change":"2023-08-11T15:44:19.867860+0000","last_active":"2023-08-11T15:44:55.194495+0000","last_peered":"2023-08-11T15:44:55.194495+0000","last_clean":"2023-08-11T15:44:55.194495+0000","last_became_active":"2023-08-11T15:44:19.867545+0000","last_became_peered":"2023-08-11T15:44:19.867545+0000","last_unstale":"2023-08-11T15:44:55.194495+0000","last_undegraded":"2023-08-11T15:44:55.194495+0000","last_fullsized":"2023-08-11T15:44:55.194495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:46:29.660791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189976+0000","last_change":"2023-08-11T15:44:24.197251+0000","last_active":"2023-08-11T15:44:55.189976+0000","last_peered":"2023-08-11T15:44:55.189976+0000","last_clean":"2023-08-11T15:44:55.189976+0000","last_became_active":"2023-08-11T15:44:24.196735+0000","last_became_peered":"2023-08-11T15:44:24.196735+0000","last_unstale":"2023-08-11T15:44:55.189976+0000","last_undegraded":"2023-08-11T15:44:55.189976+0000","last_fullsized":"2023-08-11T15:44:55.189976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:04:00.057964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191849+0000","last_change":"2023-08-11T15:44:19.146196+0000","last_active":"2023-08-11T15:44:55.191849+0000","last_peered":"2023-08-11T15:44:55.191849+0000","last_clean":"2023-08-11T15:44:55.191849+0000","last_became_active":"2023-08-11T15:44:19.145812+0000","last_became_peered":"2023-08-11T15:44:19.145812+0000","last_unstale":"2023-08-11T15:44:55.191849+0000","last_undegraded":"2023-08-11T15:44:55.191849+0000","last_fullsized":"2023-08-11T15:44:55.191849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:51:59.428704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192831+0000","last_change":"2023-08-11T15:44:19.855389+0000","last_active":"2023-08-11T15:44:55.192831+0000","last_peered":"2023-08-11T15:44:55.192831+0000","last_clean":"2023-08-11T15:44:55.192831+0000","last_became_active":"2023-08-11T15:44:19.855063+0000","last_became_peered":"2023-08-11T15:44:19.855063+0000","last_unstale":"2023-08-11T15:44:55.192831+0000","last_undegraded":"2023-08-11T15:44:55.192831+0000","last_fullsized":"2023-08-11T15:44:55.192831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:42:37.301427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194782+0000","last_change":"2023-08-11T15:44:24.192450+0000","last_active":"2023-08-11T15:44:55.194782+0000","last_peered":"2023-08-11T15:44:55.194782+0000","last_clean":"2023-08-11T15:44:55.194782+0000","last_became_active":"2023-08-11T15:44:24.191989+0000","last_became_peered":"2023-08-11T15:44:24.191989+0000","last_unstale":"2023-08-11T15:44:55.194782+0000","last_undegraded":"2023-08-11T15:44:55.194782+0000","last_fullsized":"2023-08-11T15:44:55.194782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:06.182448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192857+0000","last_change":"2023-08-11T15:44:19.149002+0000","last_active":"2023-08-11T15:44:55.192857+0000","last_peered":"2023-08-11T15:44:55.192857+0000","last_clean":"2023-08-11T15:44:55.192857+0000","last_became_active":"2023-08-11T15:44:19.148635+0000","last_became_peered":"2023-08-11T15:44:19.148635+0000","last_unstale":"2023-08-11T15:44:55.192857+0000","last_undegraded":"2023-08-11T15:44:55.192857+0000","last_fullsized":"2023-08-11T15:44:55.192857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:58.450017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191583+0000","last_change":"2023-08-11T15:44:19.858067+0000","last_active":"2023-08-11T15:44:55.191583+0000","last_peered":"2023-08-11T15:44:55.191583+0000","last_clean":"2023-08-11T15:44:55.191583+0000","last_became_active":"2023-08-11T15:44:19.856641+0000","last_became_peered":"2023-08-11T15:44:19.856641+0000","last_unstale":"2023-08-11T15:44:55.191583+0000","last_undegraded":"2023-08-11T15:44:55.191583+0000","last_fullsized":"2023-08-11T15:44:55.191583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:17.780319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192698+0000","last_change":"2023-08-11T15:44:24.196806+0000","last_active":"2023-08-11T15:44:55.192698+0000","last_peered":"2023-08-11T15:44:55.192698+0000","last_clean":"2023-08-11T15:44:55.192698+0000","last_became_active":"2023-08-11T15:44:24.195268+0000","last_became_peered":"2023-08-11T15:44:24.195268+0000","last_unstale":"2023-08-11T15:44:55.192698+0000","last_undegraded":"2023-08-11T15:44:55.192698+0000","last_fullsized":"2023-08-11T15:44:55.192698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:47:43.203448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.202786+0000","last_change":"2023-08-11T15:44:55.202786+0000","last_active":"2023-08-11T15:44:55.202786+0000","last_peered":"2023-08-11T15:44:55.202786+0000","last_clean":"2023-08-11T15:44:55.202786+0000","last_became_active":"2023-08-11T15:44:55.202409+0000","last_became_peered":"2023-08-11T15:44:55.202409+0000","last_unstale":"2023-08-11T15:44:55.202786+0000","last_undegraded":"2023-08-11T15:44:55.202786+0000","last_fullsized":"2023-08-11T15:44:55.202786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:22.000673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190025+0000","last_change":"2023-08-11T15:44:19.853881+0000","last_active":"2023-08-11T15:44:55.190025+0000","last_peered":"2023-08-11T15:44:55.190025+0000","last_clean":"2023-08-11T15:44:55.190025+0000","last_became_active":"2023-08-11T15:44:19.853139+0000","last_became_peered":"2023-08-11T15:44:19.853139+0000","last_unstale":"2023-08-11T15:44:55.190025+0000","last_undegraded":"2023-08-11T15:44:55.190025+0000","last_fullsized":"2023-08-11T15:44:55.190025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:23:44.862419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197483+0000","last_change":"2023-08-11T15:44:24.201620+0000","last_active":"2023-08-11T15:44:55.197483+0000","last_peered":"2023-08-11T15:44:55.197483+0000","last_clean":"2023-08-11T15:44:55.197483+0000","last_became_active":"2023-08-11T15:44:24.201162+0000","last_became_peered":"2023-08-11T15:44:24.201162+0000","last_unstale":"2023-08-11T15:44:55.197483+0000","last_undegraded":"2023-08-11T15:44:55.197483+0000","last_fullsized":"2023-08-11T15:44:55.197483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:06:42.850458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191954+0000","last_change":"2023-08-11T15:44:19.156899+0000","last_active":"2023-08-11T15:44:55.191954+0000","last_peered":"2023-08-11T15:44:55.191954+0000","last_clean":"2023-08-11T15:44:55.191954+0000","last_became_active":"2023-08-11T15:44:19.156643+0000","last_became_peered":"2023-08-11T15:44:19.156643+0000","last_unstale":"2023-08-11T15:44:55.191954+0000","last_undegraded":"2023-08-11T15:44:55.191954+0000","last_fullsized":"2023-08-11T15:44:55.191954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:07.395481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190798+0000","last_change":"2023-08-11T15:44:19.857828+0000","last_active":"2023-08-11T15:44:55.190798+0000","last_peered":"2023-08-11T15:44:55.190798+0000","last_clean":"2023-08-11T15:44:55.190798+0000","last_became_active":"2023-08-11T15:44:19.856781+0000","last_became_peered":"2023-08-11T15:44:19.856781+0000","last_unstale":"2023-08-11T15:44:55.190798+0000","last_undegraded":"2023-08-11T15:44:55.190798+0000","last_fullsized":"2023-08-11T15:44:55.190798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:59:52.008608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192640+0000","last_change":"2023-08-11T15:44:24.193213+0000","last_active":"2023-08-11T15:44:55.192640+0000","last_peered":"2023-08-11T15:44:55.192640+0000","last_clean":"2023-08-11T15:44:55.192640+0000","last_became_active":"2023-08-11T15:44:24.192819+0000","last_became_peered":"2023-08-11T15:44:24.192819+0000","last_unstale":"2023-08-11T15:44:55.192640+0000","last_undegraded":"2023-08-11T15:44:55.192640+0000","last_fullsized":"2023-08-11T15:44:55.192640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:21:03.566682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:45:42.299277+0000","last_change":"2023-08-11T15:44:19.149753+0000","last_active":"2023-08-11T15:45:42.299277+0000","last_peered":"2023-08-11T15:45:42.299277+0000","last_clean":"2023-08-11T15:45:42.299277+0000","last_became_active":"2023-08-11T15:44:19.149420+0000","last_became_peered":"2023-08-11T15:44:19.149420+0000","last_unstale":"2023-08-11T15:45:42.299277+0000","last_undegraded":"2023-08-11T15:45:42.299277+0000","last_fullsized":"2023-08-11T15:45:42.299277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:43:16.393961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61470,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":111,"num_read_kb":115,"num_write":30,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195024+0000","last_change":"2023-08-11T15:44:19.861342+0000","last_active":"2023-08-11T15:44:55.195024+0000","last_peered":"2023-08-11T15:44:55.195024+0000","last_clean":"2023-08-11T15:44:55.195024+0000","last_became_active":"2023-08-11T15:44:19.860978+0000","last_became_peered":"2023-08-11T15:44:19.860978+0000","last_unstale":"2023-08-11T15:44:55.195024+0000","last_undegraded":"2023-08-11T15:44:55.195024+0000","last_fullsized":"2023-08-11T15:44:55.195024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:12.643255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191794+0000","last_change":"2023-08-11T15:44:24.191504+0000","last_active":"2023-08-11T15:44:55.191794+0000","last_peered":"2023-08-11T15:44:55.191794+0000","last_clean":"2023-08-11T15:44:55.191794+0000","last_became_active":"2023-08-11T15:44:24.191127+0000","last_became_peered":"2023-08-11T15:44:24.191127+0000","last_unstale":"2023-08-11T15:44:55.191794+0000","last_undegraded":"2023-08-11T15:44:55.191794+0000","last_fullsized":"2023-08-11T15:44:55.191794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:47:16.917614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194641+0000","last_change":"2023-08-11T15:44:19.148354+0000","last_active":"2023-08-11T15:44:55.194641+0000","last_peered":"2023-08-11T15:44:55.194641+0000","last_clean":"2023-08-11T15:44:55.194641+0000","last_became_active":"2023-08-11T15:44:19.148036+0000","last_became_peered":"2023-08-11T15:44:19.148036+0000","last_unstale":"2023-08-11T15:44:55.194641+0000","last_undegraded":"2023-08-11T15:44:55.194641+0000","last_fullsized":"2023-08-11T15:44:55.194641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:06:50.544324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195295+0000","last_change":"2023-08-11T15:44:19.862409+0000","last_active":"2023-08-11T15:44:55.195295+0000","last_peered":"2023-08-11T15:44:55.195295+0000","last_clean":"2023-08-11T15:44:55.195295+0000","last_became_active":"2023-08-11T15:44:19.861359+0000","last_became_peered":"2023-08-11T15:44:19.861359+0000","last_unstale":"2023-08-11T15:44:55.195295+0000","last_undegraded":"2023-08-11T15:44:55.195295+0000","last_fullsized":"2023-08-11T15:44:55.195295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:22:15.811283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192490+0000","last_change":"2023-08-11T15:44:24.197634+0000","last_active":"2023-08-11T15:44:55.192490+0000","last_peered":"2023-08-11T15:44:55.192490+0000","last_clean":"2023-08-11T15:44:55.192490+0000","last_became_active":"2023-08-11T15:44:24.197181+0000","last_became_peered":"2023-08-11T15:44:24.197181+0000","last_unstale":"2023-08-11T15:44:55.192490+0000","last_undegraded":"2023-08-11T15:44:55.192490+0000","last_fullsized":"2023-08-11T15:44:55.192490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:24:25.505006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190865+0000","last_change":"2023-08-11T15:44:19.147550+0000","last_active":"2023-08-11T15:44:55.190865+0000","last_peered":"2023-08-11T15:44:55.190865+0000","last_clean":"2023-08-11T15:44:55.190865+0000","last_became_active":"2023-08-11T15:44:19.147299+0000","last_became_peered":"2023-08-11T15:44:19.147299+0000","last_unstale":"2023-08-11T15:44:55.190865+0000","last_undegraded":"2023-08-11T15:44:55.190865+0000","last_fullsized":"2023-08-11T15:44:55.190865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:16.012041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194683+0000","last_change":"2023-08-11T15:44:19.862664+0000","last_active":"2023-08-11T15:44:55.194683+0000","last_peered":"2023-08-11T15:44:55.194683+0000","last_clean":"2023-08-11T15:44:55.194683+0000","last_became_active":"2023-08-11T15:44:19.862130+0000","last_became_peered":"2023-08-11T15:44:19.862130+0000","last_unstale":"2023-08-11T15:44:55.194683+0000","last_undegraded":"2023-08-11T15:44:55.194683+0000","last_fullsized":"2023-08-11T15:44:55.194683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:26:44.167213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193403+0000","last_change":"2023-08-11T15:44:24.199851+0000","last_active":"2023-08-11T15:44:55.193403+0000","last_peered":"2023-08-11T15:44:55.193403+0000","last_clean":"2023-08-11T15:44:55.193403+0000","last_became_active":"2023-08-11T15:44:24.199420+0000","last_became_peered":"2023-08-11T15:44:24.199420+0000","last_unstale":"2023-08-11T15:44:55.193403+0000","last_undegraded":"2023-08-11T15:44:55.193403+0000","last_fullsized":"2023-08-11T15:44:55.193403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:32.647518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198075+0000","last_change":"2023-08-11T15:44:19.145888+0000","last_active":"2023-08-11T15:44:55.198075+0000","last_peered":"2023-08-11T15:44:55.198075+0000","last_clean":"2023-08-11T15:44:55.198075+0000","last_became_active":"2023-08-11T15:44:19.145459+0000","last_became_peered":"2023-08-11T15:44:19.145459+0000","last_unstale":"2023-08-11T15:44:55.198075+0000","last_undegraded":"2023-08-11T15:44:55.198075+0000","last_fullsized":"2023-08-11T15:44:55.198075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:29.167474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194230+0000","last_change":"2023-08-11T15:44:19.860035+0000","last_active":"2023-08-11T15:44:55.194230+0000","last_peered":"2023-08-11T15:44:55.194230+0000","last_clean":"2023-08-11T15:44:55.194230+0000","last_became_active":"2023-08-11T15:44:19.859696+0000","last_became_peered":"2023-08-11T15:44:19.859696+0000","last_unstale":"2023-08-11T15:44:55.194230+0000","last_undegraded":"2023-08-11T15:44:55.194230+0000","last_fullsized":"2023-08-11T15:44:55.194230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:33.718669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196792+0000","last_change":"2023-08-11T15:44:24.198255+0000","last_active":"2023-08-11T15:44:55.196792+0000","last_peered":"2023-08-11T15:44:55.196792+0000","last_clean":"2023-08-11T15:44:55.196792+0000","last_became_active":"2023-08-11T15:44:24.197825+0000","last_became_peered":"2023-08-11T15:44:24.197825+0000","last_unstale":"2023-08-11T15:44:55.196792+0000","last_undegraded":"2023-08-11T15:44:55.196792+0000","last_fullsized":"2023-08-11T15:44:55.196792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:00:15.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.207182+0000","last_change":"2023-08-11T15:44:55.206745+0000","last_active":"2023-08-11T15:44:55.207182+0000","last_peered":"2023-08-11T15:44:55.207182+0000","last_clean":"2023-08-11T15:44:55.207182+0000","last_became_active":"2023-08-11T15:44:55.196900+0000","last_became_peered":"2023-08-11T15:44:55.196900+0000","last_unstale":"2023-08-11T15:44:55.207182+0000","last_undegraded":"2023-08-11T15:44:55.207182+0000","last_fullsized":"2023-08-11T15:44:55.207182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:32.529218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190529+0000","last_change":"2023-08-11T15:44:19.853273+0000","last_active":"2023-08-11T15:44:55.190529+0000","last_peered":"2023-08-11T15:44:55.190529+0000","last_clean":"2023-08-11T15:44:55.190529+0000","last_became_active":"2023-08-11T15:44:19.852914+0000","last_became_peered":"2023-08-11T15:44:19.852914+0000","last_unstale":"2023-08-11T15:44:55.190529+0000","last_undegraded":"2023-08-11T15:44:55.190529+0000","last_fullsized":"2023-08-11T15:44:55.190529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:33.263048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190457+0000","last_change":"2023-08-11T15:44:24.194429+0000","last_active":"2023-08-11T15:44:55.190457+0000","last_peered":"2023-08-11T15:44:55.190457+0000","last_clean":"2023-08-11T15:44:55.190457+0000","last_became_active":"2023-08-11T15:44:24.194089+0000","last_became_peered":"2023-08-11T15:44:24.194089+0000","last_unstale":"2023-08-11T15:44:55.190457+0000","last_undegraded":"2023-08-11T15:44:55.190457+0000","last_fullsized":"2023-08-11T15:44:55.190457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:20:34.396873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195649+0000","last_change":"2023-08-11T15:44:19.151072+0000","last_active":"2023-08-11T15:44:55.195649+0000","last_peered":"2023-08-11T15:44:55.195649+0000","last_clean":"2023-08-11T15:44:55.195649+0000","last_became_active":"2023-08-11T15:44:19.147713+0000","last_became_peered":"2023-08-11T15:44:19.147713+0000","last_unstale":"2023-08-11T15:44:55.195649+0000","last_undegraded":"2023-08-11T15:44:55.195649+0000","last_fullsized":"2023-08-11T15:44:55.195649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:43.361011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192121+0000","last_change":"2023-08-11T15:44:19.862259+0000","last_active":"2023-08-11T15:44:55.192121+0000","last_peered":"2023-08-11T15:44:55.192121+0000","last_clean":"2023-08-11T15:44:55.192121+0000","last_became_active":"2023-08-11T15:44:19.861910+0000","last_became_peered":"2023-08-11T15:44:19.861910+0000","last_unstale":"2023-08-11T15:44:55.192121+0000","last_undegraded":"2023-08-11T15:44:55.192121+0000","last_fullsized":"2023-08-11T15:44:55.192121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:31.107437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198879+0000","last_change":"2023-08-11T15:44:55.198879+0000","last_active":"2023-08-11T15:44:55.198879+0000","last_peered":"2023-08-11T15:44:55.198879+0000","last_clean":"2023-08-11T15:44:55.198879+0000","last_became_active":"2023-08-11T15:44:55.198461+0000","last_became_peered":"2023-08-11T15:44:55.198461+0000","last_unstale":"2023-08-11T15:44:55.198879+0000","last_undegraded":"2023-08-11T15:44:55.198879+0000","last_fullsized":"2023-08-11T15:44:55.198879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:04:45.789628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193224+0000","last_change":"2023-08-11T15:44:19.152608+0000","last_active":"2023-08-11T15:44:55.193224+0000","last_peered":"2023-08-11T15:44:55.193224+0000","last_clean":"2023-08-11T15:44:55.193224+0000","last_became_active":"2023-08-11T15:44:19.152255+0000","last_became_peered":"2023-08-11T15:44:19.152255+0000","last_unstale":"2023-08-11T15:44:55.193224+0000","last_undegraded":"2023-08-11T15:44:55.193224+0000","last_fullsized":"2023-08-11T15:44:55.193224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:01.248200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193585+0000","last_change":"2023-08-11T15:44:19.864003+0000","last_active":"2023-08-11T15:44:55.193585+0000","last_peered":"2023-08-11T15:44:55.193585+0000","last_clean":"2023-08-11T15:44:55.193585+0000","last_became_active":"2023-08-11T15:44:19.863661+0000","last_became_peered":"2023-08-11T15:44:19.863661+0000","last_unstale":"2023-08-11T15:44:55.193585+0000","last_undegraded":"2023-08-11T15:44:55.193585+0000","last_fullsized":"2023-08-11T15:44:55.193585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:40:30.368843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196098+0000","last_change":"2023-08-11T15:44:24.194123+0000","last_active":"2023-08-11T15:44:55.196098+0000","last_peered":"2023-08-11T15:44:55.196098+0000","last_clean":"2023-08-11T15:44:55.196098+0000","last_became_active":"2023-08-11T15:44:24.193729+0000","last_became_peered":"2023-08-11T15:44:24.193729+0000","last_unstale":"2023-08-11T15:44:55.196098+0000","last_undegraded":"2023-08-11T15:44:55.196098+0000","last_fullsized":"2023-08-11T15:44:55.196098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:46:01.331338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194389+0000","last_change":"2023-08-11T15:44:19.151367+0000","last_active":"2023-08-11T15:44:55.194389+0000","last_peered":"2023-08-11T15:44:55.194389+0000","last_clean":"2023-08-11T15:44:55.194389+0000","last_became_active":"2023-08-11T15:44:19.150991+0000","last_became_peered":"2023-08-11T15:44:19.150991+0000","last_unstale":"2023-08-11T15:44:55.194389+0000","last_undegraded":"2023-08-11T15:44:55.194389+0000","last_fullsized":"2023-08-11T15:44:55.194389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:26:13.610294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192907+0000","last_change":"2023-08-11T15:44:19.858579+0000","last_active":"2023-08-11T15:44:55.192907+0000","last_peered":"2023-08-11T15:44:55.192907+0000","last_clean":"2023-08-11T15:44:55.192907+0000","last_became_active":"2023-08-11T15:44:19.858184+0000","last_became_peered":"2023-08-11T15:44:19.858184+0000","last_unstale":"2023-08-11T15:44:55.192907+0000","last_undegraded":"2023-08-11T15:44:55.192907+0000","last_fullsized":"2023-08-11T15:44:55.192907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:51:28.014074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196948+0000","last_change":"2023-08-11T15:44:24.300526+0000","last_active":"2023-08-11T15:44:55.196948+0000","last_peered":"2023-08-11T15:44:55.196948+0000","last_clean":"2023-08-11T15:44:55.196948+0000","last_became_active":"2023-08-11T15:44:24.299883+0000","last_became_peered":"2023-08-11T15:44:24.299883+0000","last_unstale":"2023-08-11T15:44:55.196948+0000","last_undegraded":"2023-08-11T15:44:55.196948+0000","last_fullsized":"2023-08-11T15:44:55.196948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:10.792495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191233+0000","last_change":"2023-08-11T15:44:19.858116+0000","last_active":"2023-08-11T15:44:55.191233+0000","last_peered":"2023-08-11T15:44:55.191233+0000","last_clean":"2023-08-11T15:44:55.191233+0000","last_became_active":"2023-08-11T15:44:19.857728+0000","last_became_peered":"2023-08-11T15:44:19.857728+0000","last_unstale":"2023-08-11T15:44:55.191233+0000","last_undegraded":"2023-08-11T15:44:55.191233+0000","last_fullsized":"2023-08-11T15:44:55.191233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:50.561098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195613+0000","last_change":"2023-08-11T15:44:19.157904+0000","last_active":"2023-08-11T15:44:55.195613+0000","last_peered":"2023-08-11T15:44:55.195613+0000","last_clean":"2023-08-11T15:44:55.195613+0000","last_became_active":"2023-08-11T15:44:19.157497+0000","last_became_peered":"2023-08-11T15:44:19.157497+0000","last_unstale":"2023-08-11T15:44:55.195613+0000","last_undegraded":"2023-08-11T15:44:55.195613+0000","last_fullsized":"2023-08-11T15:44:55.195613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:10:52.538491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194951+0000","last_change":"2023-08-11T15:44:24.201246+0000","last_active":"2023-08-11T15:44:55.194951+0000","last_peered":"2023-08-11T15:44:55.194951+0000","last_clean":"2023-08-11T15:44:55.194951+0000","last_became_active":"2023-08-11T15:44:24.200810+0000","last_became_peered":"2023-08-11T15:44:24.200810+0000","last_unstale":"2023-08-11T15:44:55.194951+0000","last_undegraded":"2023-08-11T15:44:55.194951+0000","last_fullsized":"2023-08-11T15:44:55.194951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:41:26.333082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195176+0000","last_change":"2023-08-11T15:44:19.846296+0000","last_active":"2023-08-11T15:44:55.195176+0000","last_peered":"2023-08-11T15:44:55.195176+0000","last_clean":"2023-08-11T15:44:55.195176+0000","last_became_active":"2023-08-11T15:44:19.845890+0000","last_became_peered":"2023-08-11T15:44:19.845890+0000","last_unstale":"2023-08-11T15:44:55.195176+0000","last_undegraded":"2023-08-11T15:44:55.195176+0000","last_fullsized":"2023-08-11T15:44:55.195176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:34:40.965829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197297+0000","last_change":"2023-08-11T15:44:19.144202+0000","last_active":"2023-08-11T15:44:55.197297+0000","last_peered":"2023-08-11T15:44:55.197297+0000","last_clean":"2023-08-11T15:44:55.197297+0000","last_became_active":"2023-08-11T15:44:19.143789+0000","last_became_peered":"2023-08-11T15:44:19.143789+0000","last_unstale":"2023-08-11T15:44:55.197297+0000","last_undegraded":"2023-08-11T15:44:55.197297+0000","last_fullsized":"2023-08-11T15:44:55.197297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:15.141815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193739+0000","last_change":"2023-08-11T15:43:14.593634+0000","last_active":"2023-08-11T15:44:55.193739+0000","last_peered":"2023-08-11T15:44:55.193739+0000","last_clean":"2023-08-11T15:44:55.193739+0000","last_became_active":"2023-08-11T15:43:14.593164+0000","last_became_peered":"2023-08-11T15:43:14.593164+0000","last_unstale":"2023-08-11T15:44:55.193739+0000","last_undegraded":"2023-08-11T15:44:55.193739+0000","last_fullsized":"2023-08-11T15:44:55.193739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_clean_scrub_stamp":"2023-08-11T15:40:09.009169+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:13:50.737842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190789+0000","last_change":"2023-08-11T15:44:24.300703+0000","last_active":"2023-08-11T15:44:55.190789+0000","last_peered":"2023-08-11T15:44:55.190789+0000","last_clean":"2023-08-11T15:44:55.190789+0000","last_became_active":"2023-08-11T15:44:24.300309+0000","last_became_peered":"2023-08-11T15:44:24.300309+0000","last_unstale":"2023-08-11T15:44:55.190789+0000","last_undegraded":"2023-08-11T15:44:55.190789+0000","last_fullsized":"2023-08-11T15:44:55.190789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:11.965214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201619+0000","last_change":"2023-08-11T15:44:55.201619+0000","last_active":"2023-08-11T15:44:55.201619+0000","last_peered":"2023-08-11T15:44:55.201619+0000","last_clean":"2023-08-11T15:44:55.201619+0000","last_became_active":"2023-08-11T15:44:55.201233+0000","last_became_peered":"2023-08-11T15:44:55.201233+0000","last_unstale":"2023-08-11T15:44:55.201619+0000","last_undegraded":"2023-08-11T15:44:55.201619+0000","last_fullsized":"2023-08-11T15:44:55.201619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:11:25.230480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193945+0000","last_change":"2023-08-11T15:44:19.148113+0000","last_active":"2023-08-11T15:44:55.193945+0000","last_peered":"2023-08-11T15:44:55.193945+0000","last_clean":"2023-08-11T15:44:55.193945+0000","last_became_active":"2023-08-11T15:44:19.147793+0000","last_became_peered":"2023-08-11T15:44:19.147793+0000","last_unstale":"2023-08-11T15:44:55.193945+0000","last_undegraded":"2023-08-11T15:44:55.193945+0000","last_fullsized":"2023-08-11T15:44:55.193945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:28.804554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191891+0000","last_change":"2023-08-11T15:44:24.194418+0000","last_active":"2023-08-11T15:44:55.191891+0000","last_peered":"2023-08-11T15:44:55.191891+0000","last_clean":"2023-08-11T15:44:55.191891+0000","last_became_active":"2023-08-11T15:44:24.193943+0000","last_became_peered":"2023-08-11T15:44:24.193943+0000","last_unstale":"2023-08-11T15:44:55.191891+0000","last_undegraded":"2023-08-11T15:44:55.191891+0000","last_fullsized":"2023-08-11T15:44:55.191891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:56:38.790282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195211+0000","last_change":"2023-08-11T15:44:19.151476+0000","last_active":"2023-08-11T15:44:55.195211+0000","last_peered":"2023-08-11T15:44:55.195211+0000","last_clean":"2023-08-11T15:44:55.195211+0000","last_became_active":"2023-08-11T15:44:19.151140+0000","last_became_peered":"2023-08-11T15:44:19.151140+0000","last_unstale":"2023-08-11T15:44:55.195211+0000","last_undegraded":"2023-08-11T15:44:55.195211+0000","last_fullsized":"2023-08-11T15:44:55.195211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:43.957395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196271+0000","last_change":"2023-08-11T15:44:19.862255+0000","last_active":"2023-08-11T15:44:55.196271+0000","last_peered":"2023-08-11T15:44:55.196271+0000","last_clean":"2023-08-11T15:44:55.196271+0000","last_became_active":"2023-08-11T15:44:19.861907+0000","last_became_peered":"2023-08-11T15:44:19.861907+0000","last_unstale":"2023-08-11T15:44:55.196271+0000","last_undegraded":"2023-08-11T15:44:55.196271+0000","last_fullsized":"2023-08-11T15:44:55.196271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:33:45.120706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194419+0000","last_change":"2023-08-11T15:44:24.300513+0000","last_active":"2023-08-11T15:44:55.194419+0000","last_peered":"2023-08-11T15:44:55.194419+0000","last_clean":"2023-08-11T15:44:55.194419+0000","last_became_active":"2023-08-11T15:44:24.300074+0000","last_became_peered":"2023-08-11T15:44:24.300074+0000","last_unstale":"2023-08-11T15:44:55.194419+0000","last_undegraded":"2023-08-11T15:44:55.194419+0000","last_fullsized":"2023-08-11T15:44:55.194419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:22.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:56.969234+0000","last_change":"2023-08-11T15:44:19.153794+0000","last_active":"2023-08-11T15:44:56.969234+0000","last_peered":"2023-08-11T15:44:56.969234+0000","last_clean":"2023-08-11T15:44:56.969234+0000","last_became_active":"2023-08-11T15:44:19.153392+0000","last_became_peered":"2023-08-11T15:44:19.153392+0000","last_unstale":"2023-08-11T15:44:56.969234+0000","last_undegraded":"2023-08-11T15:44:56.969234+0000","last_fullsized":"2023-08-11T15:44:56.969234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:59:18.150146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197135+0000","last_change":"2023-08-11T15:44:19.859799+0000","last_active":"2023-08-11T15:44:55.197135+0000","last_peered":"2023-08-11T15:44:55.197135+0000","last_clean":"2023-08-11T15:44:55.197135+0000","last_became_active":"2023-08-11T15:44:19.859411+0000","last_became_peered":"2023-08-11T15:44:19.859411+0000","last_unstale":"2023-08-11T15:44:55.197135+0000","last_undegraded":"2023-08-11T15:44:55.197135+0000","last_fullsized":"2023-08-11T15:44:55.197135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:50:04.069525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194744+0000","last_change":"2023-08-11T15:44:24.190272+0000","last_active":"2023-08-11T15:44:55.194744+0000","last_peered":"2023-08-11T15:44:55.194744+0000","last_clean":"2023-08-11T15:44:55.194744+0000","last_became_active":"2023-08-11T15:44:24.189849+0000","last_became_peered":"2023-08-11T15:44:24.189849+0000","last_unstale":"2023-08-11T15:44:55.194744+0000","last_undegraded":"2023-08-11T15:44:55.194744+0000","last_fullsized":"2023-08-11T15:44:55.194744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:21:45.389663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198407+0000","last_change":"2023-08-11T15:44:19.154985+0000","last_active":"2023-08-11T15:44:55.198407+0000","last_peered":"2023-08-11T15:44:55.198407+0000","last_clean":"2023-08-11T15:44:55.198407+0000","last_became_active":"2023-08-11T15:44:19.154694+0000","last_became_peered":"2023-08-11T15:44:19.154694+0000","last_unstale":"2023-08-11T15:44:55.198407+0000","last_undegraded":"2023-08-11T15:44:55.198407+0000","last_fullsized":"2023-08-11T15:44:55.198407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:22:34.297160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190129+0000","last_change":"2023-08-11T15:44:19.852247+0000","last_active":"2023-08-11T15:44:55.190129+0000","last_peered":"2023-08-11T15:44:55.190129+0000","last_clean":"2023-08-11T15:44:55.190129+0000","last_became_active":"2023-08-11T15:44:19.851456+0000","last_became_peered":"2023-08-11T15:44:19.851456+0000","last_unstale":"2023-08-11T15:44:55.190129+0000","last_undegraded":"2023-08-11T15:44:55.190129+0000","last_fullsized":"2023-08-11T15:44:55.190129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:43:25.512545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.199076+0000","last_change":"2023-08-11T15:44:24.300440+0000","last_active":"2023-08-11T15:44:55.199076+0000","last_peered":"2023-08-11T15:44:55.199076+0000","last_clean":"2023-08-11T15:44:55.199076+0000","last_became_active":"2023-08-11T15:44:24.299981+0000","last_became_peered":"2023-08-11T15:44:24.299981+0000","last_unstale":"2023-08-11T15:44:55.199076+0000","last_undegraded":"2023-08-11T15:44:55.199076+0000","last_fullsized":"2023-08-11T15:44:55.199076+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:10:48.146770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190574+0000","last_change":"2023-08-11T15:44:19.151015+0000","last_active":"2023-08-11T15:44:55.190574+0000","last_peered":"2023-08-11T15:44:55.190574+0000","last_clean":"2023-08-11T15:44:55.190574+0000","last_became_active":"2023-08-11T15:44:19.150677+0000","last_became_peered":"2023-08-11T15:44:19.150677+0000","last_unstale":"2023-08-11T15:44:55.190574+0000","last_undegraded":"2023-08-11T15:44:55.190574+0000","last_fullsized":"2023-08-11T15:44:55.190574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:44.956238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194239+0000","last_change":"2023-08-11T15:44:19.861033+0000","last_active":"2023-08-11T15:44:55.194239+0000","last_peered":"2023-08-11T15:44:55.194239+0000","last_clean":"2023-08-11T15:44:55.194239+0000","last_became_active":"2023-08-11T15:44:19.860643+0000","last_became_peered":"2023-08-11T15:44:19.860643+0000","last_unstale":"2023-08-11T15:44:55.194239+0000","last_undegraded":"2023-08-11T15:44:55.194239+0000","last_fullsized":"2023-08-11T15:44:55.194239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:32:25.731474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193496+0000","last_change":"2023-08-11T15:44:24.199389+0000","last_active":"2023-08-11T15:44:55.193496+0000","last_peered":"2023-08-11T15:44:55.193496+0000","last_clean":"2023-08-11T15:44:55.193496+0000","last_became_active":"2023-08-11T15:44:24.196226+0000","last_became_peered":"2023-08-11T15:44:24.196226+0000","last_unstale":"2023-08-11T15:44:55.193496+0000","last_undegraded":"2023-08-11T15:44:55.193496+0000","last_fullsized":"2023-08-11T15:44:55.193496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:05:34.885169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194914+0000","last_change":"2023-08-11T15:44:19.152895+0000","last_active":"2023-08-11T15:44:55.194914+0000","last_peered":"2023-08-11T15:44:55.194914+0000","last_clean":"2023-08-11T15:44:55.194914+0000","last_became_active":"2023-08-11T15:44:19.152577+0000","last_became_peered":"2023-08-11T15:44:19.152577+0000","last_unstale":"2023-08-11T15:44:55.194914+0000","last_undegraded":"2023-08-11T15:44:55.194914+0000","last_fullsized":"2023-08-11T15:44:55.194914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:23:00.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192632+0000","last_change":"2023-08-11T15:44:19.861101+0000","last_active":"2023-08-11T15:44:55.192632+0000","last_peered":"2023-08-11T15:44:55.192632+0000","last_clean":"2023-08-11T15:44:55.192632+0000","last_became_active":"2023-08-11T15:44:19.860529+0000","last_became_peered":"2023-08-11T15:44:19.860529+0000","last_unstale":"2023-08-11T15:44:55.192632+0000","last_undegraded":"2023-08-11T15:44:55.192632+0000","last_fullsized":"2023-08-11T15:44:55.192632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:30:49.161446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197970+0000","last_change":"2023-08-11T15:44:24.202856+0000","last_active":"2023-08-11T15:44:55.197970+0000","last_peered":"2023-08-11T15:44:55.197970+0000","last_clean":"2023-08-11T15:44:55.197970+0000","last_became_active":"2023-08-11T15:44:24.202317+0000","last_became_peered":"2023-08-11T15:44:24.202317+0000","last_unstale":"2023-08-11T15:44:55.197970+0000","last_undegraded":"2023-08-11T15:44:55.197970+0000","last_fullsized":"2023-08-11T15:44:55.197970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:41:10.081275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195594+0000","last_change":"2023-08-11T15:44:19.148711+0000","last_active":"2023-08-11T15:44:55.195594+0000","last_peered":"2023-08-11T15:44:55.195594+0000","last_clean":"2023-08-11T15:44:55.195594+0000","last_became_active":"2023-08-11T15:44:19.148361+0000","last_became_peered":"2023-08-11T15:44:19.148361+0000","last_unstale":"2023-08-11T15:44:55.195594+0000","last_undegraded":"2023-08-11T15:44:55.195594+0000","last_fullsized":"2023-08-11T15:44:55.195594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:05:10.590173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195941+0000","last_change":"2023-08-11T15:44:19.859562+0000","last_active":"2023-08-11T15:44:55.195941+0000","last_peered":"2023-08-11T15:44:55.195941+0000","last_clean":"2023-08-11T15:44:55.195941+0000","last_became_active":"2023-08-11T15:44:19.859243+0000","last_became_peered":"2023-08-11T15:44:19.859243+0000","last_unstale":"2023-08-11T15:44:55.195941+0000","last_undegraded":"2023-08-11T15:44:55.195941+0000","last_fullsized":"2023-08-11T15:44:55.195941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:50:37.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197923+0000","last_change":"2023-08-11T15:44:24.300317+0000","last_active":"2023-08-11T15:44:55.197923+0000","last_peered":"2023-08-11T15:44:55.197923+0000","last_clean":"2023-08-11T15:44:55.197923+0000","last_became_active":"2023-08-11T15:44:24.299841+0000","last_became_peered":"2023-08-11T15:44:24.299841+0000","last_unstale":"2023-08-11T15:44:55.197923+0000","last_undegraded":"2023-08-11T15:44:55.197923+0000","last_fullsized":"2023-08-11T15:44:55.197923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:39.799187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194316+0000","last_change":"2023-08-11T15:44:19.149111+0000","last_active":"2023-08-11T15:44:55.194316+0000","last_peered":"2023-08-11T15:44:55.194316+0000","last_clean":"2023-08-11T15:44:55.194316+0000","last_became_active":"2023-08-11T15:44:19.148173+0000","last_became_peered":"2023-08-11T15:44:19.148173+0000","last_unstale":"2023-08-11T15:44:55.194316+0000","last_undegraded":"2023-08-11T15:44:55.194316+0000","last_fullsized":"2023-08-11T15:44:55.194316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:13:23.513921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194513+0000","last_change":"2023-08-11T15:44:19.863724+0000","last_active":"2023-08-11T15:44:55.194513+0000","last_peered":"2023-08-11T15:44:55.194513+0000","last_clean":"2023-08-11T15:44:55.194513+0000","last_became_active":"2023-08-11T15:44:19.863268+0000","last_became_peered":"2023-08-11T15:44:19.863268+0000","last_unstale":"2023-08-11T15:44:55.194513+0000","last_undegraded":"2023-08-11T15:44:55.194513+0000","last_fullsized":"2023-08-11T15:44:55.194513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:59.448457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190468+0000","last_change":"2023-08-11T15:44:24.191301+0000","last_active":"2023-08-11T15:44:55.190468+0000","last_peered":"2023-08-11T15:44:55.190468+0000","last_clean":"2023-08-11T15:44:55.190468+0000","last_became_active":"2023-08-11T15:44:24.190686+0000","last_became_peered":"2023-08-11T15:44:24.190686+0000","last_unstale":"2023-08-11T15:44:55.190468+0000","last_undegraded":"2023-08-11T15:44:55.190468+0000","last_fullsized":"2023-08-11T15:44:55.190468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:26:07.780061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195770+0000","last_change":"2023-08-11T15:44:19.147756+0000","last_active":"2023-08-11T15:44:55.195770+0000","last_peered":"2023-08-11T15:44:55.195770+0000","last_clean":"2023-08-11T15:44:55.195770+0000","last_became_active":"2023-08-11T15:44:19.147426+0000","last_became_peered":"2023-08-11T15:44:19.147426+0000","last_unstale":"2023-08-11T15:44:55.195770+0000","last_undegraded":"2023-08-11T15:44:55.195770+0000","last_fullsized":"2023-08-11T15:44:55.195770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:17:56.074888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195206+0000","last_change":"2023-08-11T15:44:19.863354+0000","last_active":"2023-08-11T15:44:55.195206+0000","last_peered":"2023-08-11T15:44:55.195206+0000","last_clean":"2023-08-11T15:44:55.195206+0000","last_became_active":"2023-08-11T15:44:19.862700+0000","last_became_peered":"2023-08-11T15:44:19.862700+0000","last_unstale":"2023-08-11T15:44:55.195206+0000","last_undegraded":"2023-08-11T15:44:55.195206+0000","last_fullsized":"2023-08-11T15:44:55.195206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:17:30.462182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189633+0000","last_change":"2023-08-11T15:44:24.204657+0000","last_active":"2023-08-11T15:44:55.189633+0000","last_peered":"2023-08-11T15:44:55.189633+0000","last_clean":"2023-08-11T15:44:55.189633+0000","last_became_active":"2023-08-11T15:44:24.204108+0000","last_became_peered":"2023-08-11T15:44:24.204108+0000","last_unstale":"2023-08-11T15:44:55.189633+0000","last_undegraded":"2023-08-11T15:44:55.189633+0000","last_fullsized":"2023-08-11T15:44:55.189633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:25.563864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191508+0000","last_change":"2023-08-11T15:44:19.147014+0000","last_active":"2023-08-11T15:44:55.191508+0000","last_peered":"2023-08-11T15:44:55.191508+0000","last_clean":"2023-08-11T15:44:55.191508+0000","last_became_active":"2023-08-11T15:44:19.146720+0000","last_became_peered":"2023-08-11T15:44:19.146720+0000","last_unstale":"2023-08-11T15:44:55.191508+0000","last_undegraded":"2023-08-11T15:44:55.191508+0000","last_fullsized":"2023-08-11T15:44:55.191508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:44:30.278828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201693+0000","last_change":"2023-08-11T15:44:55.201693+0000","last_active":"2023-08-11T15:44:55.201693+0000","last_peered":"2023-08-11T15:44:55.201693+0000","last_clean":"2023-08-11T15:44:55.201693+0000","last_became_active":"2023-08-11T15:44:55.201360+0000","last_became_peered":"2023-08-11T15:44:55.201360+0000","last_unstale":"2023-08-11T15:44:55.201693+0000","last_undegraded":"2023-08-11T15:44:55.201693+0000","last_fullsized":"2023-08-11T15:44:55.201693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:54:43.667678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200415+0000","last_change":"2023-08-11T15:44:55.197220+0000","last_active":"2023-08-11T15:44:55.200415+0000","last_peered":"2023-08-11T15:44:55.200415+0000","last_clean":"2023-08-11T15:44:55.200415+0000","last_became_active":"2023-08-11T15:44:55.196808+0000","last_became_peered":"2023-08-11T15:44:55.196808+0000","last_unstale":"2023-08-11T15:44:55.200415+0000","last_undegraded":"2023-08-11T15:44:55.200415+0000","last_fullsized":"2023-08-11T15:44:55.200415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:42:42.724543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196765+0000","last_change":"2023-08-11T15:44:19.158301+0000","last_active":"2023-08-11T15:44:55.196765+0000","last_peered":"2023-08-11T15:44:55.196765+0000","last_clean":"2023-08-11T15:44:55.196765+0000","last_became_active":"2023-08-11T15:44:19.157990+0000","last_became_peered":"2023-08-11T15:44:19.157990+0000","last_unstale":"2023-08-11T15:44:55.196765+0000","last_undegraded":"2023-08-11T15:44:55.196765+0000","last_fullsized":"2023-08-11T15:44:55.196765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:47:15.846659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195216+0000","last_change":"2023-08-11T15:44:19.857357+0000","last_active":"2023-08-11T15:44:55.195216+0000","last_peered":"2023-08-11T15:44:55.195216+0000","last_clean":"2023-08-11T15:44:55.195216+0000","last_became_active":"2023-08-11T15:44:19.857020+0000","last_became_peered":"2023-08-11T15:44:19.857020+0000","last_unstale":"2023-08-11T15:44:55.195216+0000","last_undegraded":"2023-08-11T15:44:55.195216+0000","last_fullsized":"2023-08-11T15:44:55.195216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:00.529783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192767+0000","last_change":"2023-08-11T15:44:19.863314+0000","last_active":"2023-08-11T15:44:55.192767+0000","last_peered":"2023-08-11T15:44:55.192767+0000","last_clean":"2023-08-11T15:44:55.192767+0000","last_became_active":"2023-08-11T15:44:19.862964+0000","last_became_peered":"2023-08-11T15:44:19.862964+0000","last_unstale":"2023-08-11T15:44:55.192767+0000","last_undegraded":"2023-08-11T15:44:55.192767+0000","last_fullsized":"2023-08-11T15:44:55.192767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:45:00.154066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195297+0000","last_change":"2023-08-11T15:44:19.850575+0000","last_active":"2023-08-11T15:44:55.195297+0000","last_peered":"2023-08-11T15:44:55.195297+0000","last_clean":"2023-08-11T15:44:55.195297+0000","last_became_active":"2023-08-11T15:44:19.850231+0000","last_became_peered":"2023-08-11T15:44:19.850231+0000","last_unstale":"2023-08-11T15:44:55.195297+0000","last_undegraded":"2023-08-11T15:44:55.195297+0000","last_fullsized":"2023-08-11T15:44:55.195297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:37.529897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195464+0000","last_change":"2023-08-11T15:44:19.857387+0000","last_active":"2023-08-11T15:44:55.195464+0000","last_peered":"2023-08-11T15:44:55.195464+0000","last_clean":"2023-08-11T15:44:55.195464+0000","last_became_active":"2023-08-11T15:44:19.856830+0000","last_became_peered":"2023-08-11T15:44:19.856830+0000","last_unstale":"2023-08-11T15:44:55.195464+0000","last_undegraded":"2023-08-11T15:44:55.195464+0000","last_fullsized":"2023-08-11T15:44:55.195464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:18:48.525612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189975+0000","last_change":"2023-08-11T15:44:19.868112+0000","last_active":"2023-08-11T15:44:55.189975+0000","last_peered":"2023-08-11T15:44:55.189975+0000","last_clean":"2023-08-11T15:44:55.189975+0000","last_became_active":"2023-08-11T15:44:19.867610+0000","last_became_peered":"2023-08-11T15:44:19.867610+0000","last_unstale":"2023-08-11T15:44:55.189975+0000","last_undegraded":"2023-08-11T15:44:55.189975+0000","last_fullsized":"2023-08-11T15:44:55.189975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:13:03.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196422+0000","last_change":"2023-08-11T15:44:19.858349+0000","last_active":"2023-08-11T15:44:55.196422+0000","last_peered":"2023-08-11T15:44:55.196422+0000","last_clean":"2023-08-11T15:44:55.196422+0000","last_became_active":"2023-08-11T15:44:19.858041+0000","last_became_peered":"2023-08-11T15:44:19.858041+0000","last_unstale":"2023-08-11T15:44:55.196422+0000","last_undegraded":"2023-08-11T15:44:55.196422+0000","last_fullsized":"2023-08-11T15:44:55.196422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:52.727062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192538+0000","last_change":"2023-08-11T15:44:19.865383+0000","last_active":"2023-08-11T15:44:55.192538+0000","last_peered":"2023-08-11T15:44:55.192538+0000","last_clean":"2023-08-11T15:44:55.192538+0000","last_became_active":"2023-08-11T15:44:19.865065+0000","last_became_peered":"2023-08-11T15:44:19.865065+0000","last_unstale":"2023-08-11T15:44:55.192538+0000","last_undegraded":"2023-08-11T15:44:55.192538+0000","last_fullsized":"2023-08-11T15:44:55.192538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:28:38.454441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195185+0000","last_change":"2023-08-11T15:44:19.859297+0000","last_active":"2023-08-11T15:44:55.195185+0000","last_peered":"2023-08-11T15:44:55.195185+0000","last_clean":"2023-08-11T15:44:55.195185+0000","last_became_active":"2023-08-11T15:44:19.857517+0000","last_became_peered":"2023-08-11T15:44:19.857517+0000","last_unstale":"2023-08-11T15:44:55.195185+0000","last_undegraded":"2023-08-11T15:44:55.195185+0000","last_fullsized":"2023-08-11T15:44:55.195185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:03:57.202219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201849+0000","last_change":"2023-08-11T15:44:55.201849+0000","last_active":"2023-08-11T15:44:55.201849+0000","last_peered":"2023-08-11T15:44:55.201849+0000","last_clean":"2023-08-11T15:44:55.201849+0000","last_became_active":"2023-08-11T15:44:55.201480+0000","last_became_peered":"2023-08-11T15:44:55.201480+0000","last_unstale":"2023-08-11T15:44:55.201849+0000","last_undegraded":"2023-08-11T15:44:55.201849+0000","last_fullsized":"2023-08-11T15:44:55.201849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:57.682087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190486+0000","last_change":"2023-08-11T15:44:19.861953+0000","last_active":"2023-08-11T15:44:55.190486+0000","last_peered":"2023-08-11T15:44:55.190486+0000","last_clean":"2023-08-11T15:44:55.190486+0000","last_became_active":"2023-08-11T15:44:19.861523+0000","last_became_peered":"2023-08-11T15:44:19.861523+0000","last_unstale":"2023-08-11T15:44:55.190486+0000","last_undegraded":"2023-08-11T15:44:55.190486+0000","last_fullsized":"2023-08-11T15:44:55.190486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:22:05.760149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194869+0000","last_change":"2023-08-11T15:44:19.849283+0000","last_active":"2023-08-11T15:44:55.194869+0000","last_peered":"2023-08-11T15:44:55.194869+0000","last_clean":"2023-08-11T15:44:55.194869+0000","last_became_active":"2023-08-11T15:44:19.848304+0000","last_became_peered":"2023-08-11T15:44:19.848304+0000","last_unstale":"2023-08-11T15:44:55.194869+0000","last_undegraded":"2023-08-11T15:44:55.194869+0000","last_fullsized":"2023-08-11T15:44:55.194869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:03.496778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190770+0000","last_change":"2023-08-11T15:44:19.853532+0000","last_active":"2023-08-11T15:44:55.190770+0000","last_peered":"2023-08-11T15:44:55.190770+0000","last_clean":"2023-08-11T15:44:55.190770+0000","last_became_active":"2023-08-11T15:44:19.853182+0000","last_became_peered":"2023-08-11T15:44:19.853182+0000","last_unstale":"2023-08-11T15:44:55.190770+0000","last_undegraded":"2023-08-11T15:44:55.190770+0000","last_fullsized":"2023-08-11T15:44:55.190770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:39:40.398308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195608+0000","last_change":"2023-08-11T15:44:19.860571+0000","last_active":"2023-08-11T15:44:55.195608+0000","last_peered":"2023-08-11T15:44:55.195608+0000","last_clean":"2023-08-11T15:44:55.195608+0000","last_became_active":"2023-08-11T15:44:19.858570+0000","last_became_peered":"2023-08-11T15:44:19.858570+0000","last_unstale":"2023-08-11T15:44:55.195608+0000","last_undegraded":"2023-08-11T15:44:55.195608+0000","last_fullsized":"2023-08-11T15:44:55.195608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:26:39.509633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192077+0000","last_change":"2023-08-11T15:44:19.865988+0000","last_active":"2023-08-11T15:44:55.192077+0000","last_peered":"2023-08-11T15:44:55.192077+0000","last_clean":"2023-08-11T15:44:55.192077+0000","last_became_active":"2023-08-11T15:44:19.865613+0000","last_became_peered":"2023-08-11T15:44:19.865613+0000","last_unstale":"2023-08-11T15:44:55.192077+0000","last_undegraded":"2023-08-11T15:44:55.192077+0000","last_fullsized":"2023-08-11T15:44:55.192077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:46:16.828564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62268,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":121,"num_write":73,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":186804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":490303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.626,"15min":0.627},"min":{"1min":0.40799999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.496}]},{"osd":1,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83899999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72799999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.78100000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.442},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.29099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69899999999999995,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.54300000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85699999999999998,"5min":1.119,"15min":1.119},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91300000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.41199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.88300000000000001,"5min":1.367,"15min":1.367},"last":0.378},{"interface":"front","average":{"1min":0.626,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90300000000000002,"5min":1.631,"15min":1.631},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.4929999999999999,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79100000000000004,"15min":0.80200000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.998,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95799999999999996,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Aug 11 15:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73499999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.90000000000000002,"15min":0.91400000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0229999999999999,"5min":1.742,"15min":1.742},"last":0.752},{"interface":"front","average":{"1min":1.1930000000000001,"5min":0.94599999999999995,"15min":0.90500000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":6.7590000000000003,"5min":6.7590000000000003,"15min":6.7590000000000003},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79700000000000004,"15min":0.79800000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":1.272,"15min":1.272},"last":0.621},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82699999999999996,"15min":0.82999999999999996},"min":{"1min":0.55000000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97199999999999998,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.71399999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1137908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.621},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73699999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.753},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":1.252,"15min":1.252},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.47099999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88200000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.745,"15min":0.755},"min":{"1min":0.52500000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84399999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95599999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79300000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77300000000000002,"15min":0.77500000000000002},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.92500000000000004}]},{"osd":9,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1140000000000001,"5min":1.226,"15min":1.226},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0840000000000001,"5min":1.484,"15min":1.484},"last":0.96599999999999997}]},{"osd":11,"last update":"Fri Aug 11 15:45:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.746},"min":{"1min":0.68400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.78900000000000003,"15min":0.78000000000000003},"min":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97999999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":494981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":3.2280000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.1919999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":3.3069999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.095,"5min":1.242,"15min":1.242},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.80600000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0720000000000001,"5min":1.226,"15min":1.226},"last":3.177}]},{"osd":3,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0680000000000001,"5min":1.171,"15min":1.171},"last":0.96299999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.24,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":3.3700000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.86799999999999999,"15min":0.88700000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0449999999999999,"5min":1.504,"15min":1.504},"last":1.022}]},{"osd":5,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.262,"5min":1.371,"15min":1.371},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.875,"15min":0.88900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1859999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.749}]},{"osd":6,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.046},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1459999999999999,"5min":1.579,"15min":1.579},"last":0.91100000000000003}]},{"osd":8,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.95799999999999996,"15min":0.97199999999999998},"min":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1679999999999999,"5min":1.323,"15min":1.323},"last":3.27},{"interface":"front","average":{"1min":0.88,"5min":0.995,"15min":1.014},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3149999999999999,"5min":1.429,"15min":1.429},"last":1.169}]},{"osd":10,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.95299999999999996,"15min":0.97099999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2450000000000001,"5min":1.599,"15min":1.599},"last":3.339},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.98899999999999999,"15min":1.0029999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.081,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.302}]},{"osd":11,"last update":"Fri Aug 11 15:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.3260000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.878}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":490576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.60899999999999999,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":6.9880000000000004,"5min":6.9880000000000004,"15min":6.9880000000000004},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.627,"15min":0.61199999999999999},"min":{"1min":0.49099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.46100000000000002,"15min":0.437},"min":{"1min":0.39300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.48899999999999999,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.68200000000000005,"15min":0.63500000000000001},"min":{"1min":0.57599999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.73199999999999998}]},{"osd":4,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65100000000000002,"15min":0.63800000000000001},"min":{"1min":0.54900000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0980000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.90300000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.72299999999999998,"15min":0.70999999999999996},"min":{"1min":0.55200000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":0.81100000000000005,"15min":0.745},"min":{"1min":0.64200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":7.1079999999999997,"5min":7.1079999999999997,"15min":7.1079999999999997},"last":0.95299999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.71899999999999997,"15min":0.69699999999999995},"min":{"1min":0.57599999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.70999999999999996},"min":{"1min":0.59599999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.94299999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.90000000000000002,"15min":0.91500000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.083,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.86799999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.74299999999999999,"15min":0.72399999999999998},"min":{"1min":0.60099999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.82399999999999995}]},{"osd":11,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.90000000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1085349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.441},"min":{"1min":0.311,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71099999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.44900000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.626},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.436,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77800000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76100000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.5}]},{"osd":5,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84199999999999997,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88800000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.50900000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.79400000000000004}]},{"osd":8,"last update":"Fri Aug 11 15:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97899999999999998,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.98699999999999999,"5min":4.8890000000000002,"15min":4.8890000000000002},"last":0.78000000000000003}]},{"osd":9,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.59299999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.81000000000000005}]},{"osd":10,"last update":"Fri Aug 11 15:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67200000000000004}]},{"osd":11,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83699999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":486207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":2.077,"15min":2.077},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.008,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0620000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.581,"15min":1.581},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92500000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.752}]},{"osd":4,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.7,"15min":1.7},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0269999999999999,"5min":1.22,"15min":1.22},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1319999999999999,"5min":1.623,"15min":1.623},"last":1.111}]},{"osd":7,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0740000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.099},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.82599999999999996,"15min":0.79900000000000004},"min":{"1min":0.67000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.099,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0820000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.879,"15min":0.872},"min":{"1min":0.69999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1220000000000001,"5min":1.252,"15min":1.252},"last":1.127},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.76300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1559999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.1539999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96199999999999997,"15min":0.97399999999999998},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.121,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.98999999999999999,"15min":1},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1559999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.1819999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:45:12 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.80400000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.1839999999999999,"5min":1.274,"15min":1.274},"last":1.165},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.97099999999999997,"15min":0.96799999999999997},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.2150000000000001,"5min":1.768,"15min":1.768},"last":1.1739999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.028}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":547677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.753},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91700000000000004,"5min":4.9020000000000001,"15min":4.9020000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.318,"15min":0.318},"max":{"1min":0.86499999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59599999999999997,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94399999999999995,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.314,"15min":0.314},"max":{"1min":0.78500000000000003,"5min":1.593,"15min":1.593},"last":0.85899999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.753},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.024,"15min":1.024},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:45:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44,"15min":0.41799999999999998},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67800000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.435},"min":{"1min":0.371,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.627,"15min":0.58899999999999997},"min":{"1min":0.58099999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.621,"15min":0.58299999999999996},"min":{"1min":0.59099999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":1.05,"15min":1.05},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:45:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51100000000000001,"15min":0.48699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63400000000000001,"15min":0.60199999999999998},"min":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.88100000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.70899999999999996,"15min":0.67700000000000005},"min":{"1min":0.61399999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.56899999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Aug 11 15:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1180000000000001,"5min":1.298,"15min":1.298},"last":0.95599999999999996}]},{"osd":10,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64500000000000002,"15min":0.63800000000000001},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.94099999999999995}]},{"osd":11,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97399999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":486297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70099999999999996,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.874,"5min":6.0609999999999999,"15min":6.0609999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64200000000000002,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88400000000000001,"5min":6.101,"15min":6.101},"last":0.754}]},{"osd":2,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.56699999999999995},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93400000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.83499999999999996,"15min":0.747},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.7640000000000002,"5min":6.7640000000000002,"15min":6.7640000000000002},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.622},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":1.26,"15min":1.26},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65900000000000003,"15min":0.60499999999999998},"min":{"1min":0.496,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.056,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.872}]},{"osd":5,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.71299999999999997,"15min":0.66100000000000003},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.75,"15min":0.70799999999999996},"min":{"1min":0.64900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.032,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:45:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.61799999999999999}]},{"osd":8,"last update":"Fri Aug 11 15:45:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":0.96099999999999997,"15min":0.91200000000000003},"min":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":6.798,"5min":6.798,"15min":6.798},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.002,"5min":0.89000000000000001,"15min":0.88100000000000001},"min":{"1min":0.78100000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97799999999999998,"5min":1.339,"15min":1.339},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.98499999999999999,"15min":1.0129999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.194,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77800000000000002}]},{"osd":10,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.97299999999999998,"15min":0.93899999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":6.8159999999999998,"5min":6.8159999999999998,"15min":6.8159999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.99299999999999999,"15min":1.008},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.234,"5min":1.472,"15min":1.472},"last":0.84299999999999997}]},{"osd":11,"last update":"Fri Aug 11 15:45:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.024,"15min":0.95399999999999996},"min":{"1min":0.73199999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.91400000000000003,"15min":0.88200000000000001},"min":{"1min":0.83399999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.88800000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":547711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57499999999999996,"15min":0.54600000000000004},"min":{"1min":0.48799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.48599999999999999,"15min":0.45000000000000001},"min":{"1min":0.379,"5min":0.251,"15min":0.251},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.501,"15min":0.47299999999999998},"min":{"1min":0.33000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496}]},{"osd":6,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.75700000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.621},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.625,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.84099999999999997}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.63,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.67700000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.86799999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.91400000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":486311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.47699999999999998,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68100000000000005,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.66200000000000003}]},{"osd":4,"last update":"Fri Aug 11 15:45:07 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.752,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":5.8659999999999997,"5min":5.8659999999999997,"15min":5.8659999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.66400000000000003},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76500000000000001,"15min":0.747},"min":{"1min":0.64600000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0700000000000001,"5min":1.179,"15min":1.179},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.72399999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78100000000000003,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.5,"15min":0.5},"max":{"1min":3.9060000000000001,"5min":3.9060000000000001,"15min":3.9060000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.77400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.036,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82999999999999996}]},{"osd":8,"last update":"Fri Aug 11 15:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.97599999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:44:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95099999999999996,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.266,"5min":5.266,"15min":5.266},"last":0.753},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.89100000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89300000000000002,"15min":0.90400000000000003},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.99299999999999999,"15min":0.98799999999999999},"min":{"1min":0.73099999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.874}]},{"osd":11,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.83999999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.379},"min":{"1min":0.28299999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.53600000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.38200000000000001},"min":{"1min":0.29399999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56200000000000006,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":1.516,"15min":1.516},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82099999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.47399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81200000000000006,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.874,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89100000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83599999999999997,"5min":1.552,"15min":1.552},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":1.502,"15min":1.502},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.41899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.84399999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78900000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Aug 11 15:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89500000000000002,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.749,"15min":0.73999999999999999},"min":{"1min":0.58899999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92800000000000005,"5min":1.526,"15min":1.526},"last":0.73799999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77100000000000002,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.94099999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92100000000000004,"5min":1.024,"15min":1.024},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95499999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.754}]},{"osd":11,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.82499999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1080522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61599999999999999,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":5.359,"15min":5.359},"last":0.73399999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.92500000000000004,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.91000000000000003,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.377},"min":{"1min":0.35799999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.44,"15min":0.41099999999999998},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.55300000000000005},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61199999999999999,"15min":0.57999999999999996},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60499999999999998,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.032,"15min":1.032},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Aug 11 15:45:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.875},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.65400000000000003,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.88700000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60899999999999999,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86799999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67600000000000005,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.376,"15min":0.376},"max":{"1min":0.93799999999999994,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.83399999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.91100000000000003}]},{"osd":10,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.752,"15min":0.73399999999999999},"min":{"1min":0.72199999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.75,"15min":0.73199999999999998},"min":{"1min":0.68700000000000006,"5min":0.505,"15min":0.505},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:45:45.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph pg dump --format=json 2023-08-11T15:45:45.439 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:45 smithi032 ceph-mon[97103]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:45 smithi134 ceph-mon[130447]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:45.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:45 smithi192 ceph-mon[130806]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:46.425 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:46 smithi032 ceph-mon[97103]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:46.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:46 smithi134 ceph-mon[130447]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:46.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:46 smithi192 ceph-mon[130806]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:47.283 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:47 smithi032 ceph-mon[97103]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:47.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:47 smithi134 ceph-mon[130447]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:47.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 -- 172.21.15.32:0/3484477635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8100940 msgr2=0x7f52e8100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:47.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/3484477635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8100940 0x7f52e8100dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009990 tx=0x7f52d8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 -- 172.21.15.32:0/3484477635 shutdown_connections 2023-08-11T15:45:47.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/3484477635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e8101300 0x7f52e810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/3484477635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8106940 0x7f52e8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/3484477635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8100940 0x7f52e8100dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 -- 172.21.15.32:0/3484477635 >> 172.21.15.32:0/3484477635 conn(0x7f52e80fc410 msgr2=0x7f52e80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:47.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 -- 172.21.15.32:0/3484477635 shutdown_connections 2023-08-11T15:45:47.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.535+0000 7f52ef9a8700 1 -- 172.21.15.32:0/3484477635 wait complete. 2023-08-11T15:45:47.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.536+0000 7f52ef9a8700 1 Processor -- start 2023-08-11T15:45:47.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ef9a8700 1 -- start start 2023-08-11T15:45:47.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ef9a8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100940 0x7f52e819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:47.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ef9a8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e8101300 0x7f52e819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:47.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ef9a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8106940 0x7f52e8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:47.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ef9a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8199130 con 0x7f52e8106940 2023-08-11T15:45:47.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ef9a8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81992a0 con 0x7f52e8101300 2023-08-11T15:45:47.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ef9a8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8199460 con 0x7f52e8100940 2023-08-11T15:45:47.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ed744700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100940 0x7f52e819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:47.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52edf45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8106940 0x7f52e8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:47.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ed744700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100940 0x7f52e819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:60594/0 (socket says 172.21.15.32:60594) 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ed744700 1 -- 172.21.15.32:0/1067488094 learned_addr learned my addr 172.21.15.32:0/1067488094 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.537+0000 7f52ecf43700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e8101300 0x7f52e819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ed744700 1 -- 172.21.15.32:0/1067488094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e8101300 msgr2=0x7f52e819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ed744700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e8101300 0x7f52e819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ed744700 1 -- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8106940 msgr2=0x7f52e8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ed744700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8106940 0x7f52e8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ed744700 1 -- 172.21.15.32:0/1067488094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80095f0 con 0x7f52e8100940 2023-08-11T15:45:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ed744700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100940 0x7f52e819ea40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e9a0 tx=0x7f52e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:47.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52de7fc700 1 -- 172.21.15.32:0/1067488094 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4009790 con 0x7f52e8100940 2023-08-11T15:45:47.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e8199740 con 0x7f52e8100940 2023-08-11T15:45:47.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52de7fc700 1 -- 172.21.15.32:0/1067488094 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e400cbd0 con 0x7f52e8100940 2023-08-11T15:45:47.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52de7fc700 1 -- 172.21.15.32:0/1067488094 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4018680 con 0x7f52e8100940 2023-08-11T15:45:47.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.538+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e81a5a30 con 0x7f52e8100940 2023-08-11T15:45:47.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.539+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e804e620 con 0x7f52e8100940 2023-08-11T15:45:47.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.542+0000 7f52de7fc700 1 -- 172.21.15.32:0/1067488094 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f52e400cd40 con 0x7f52e8100940 2023-08-11T15:45:47.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.543+0000 7f52de7fc700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d409ffe0 0x7f52d40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:47.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.543+0000 7f52de7fc700 1 -- 172.21.15.32:0/1067488094 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52e4014070 con 0x7f52e8100940 2023-08-11T15:45:47.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.543+0000 7f52ecf43700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d409ffe0 0x7f52d40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:47.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.544+0000 7f52ecf43700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d409ffe0 0x7f52d40a24a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b2e0 tx=0x7f52d8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:47.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.545+0000 7f52de7fc700 1 -- 172.21.15.32:0/1067488094 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f52e4095330 con 0x7f52e8100940 2023-08-11T15:45:47.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:47 smithi192 ceph-mon[130806]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.746+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52e8110af0 con 0x7f52d409ffe0 2023-08-11T15:45:47.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.759+0000 7f52de7fc700 1 -- 172.21.15.32:0/1067488094 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426365 (secure 0 0 0) 0x7f52e8110af0 con 0x7f52d409ffe0 2023-08-11T15:45:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d409ffe0 msgr2=0x7f52d40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d409ffe0 0x7f52d40a24a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b2e0 tx=0x7f52d8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100940 msgr2=0x7f52e819ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100940 0x7f52e819ea40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e9a0 tx=0x7f52e400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 shutdown_connections 2023-08-11T15:45:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d409ffe0 0x7f52d40a24a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8106940 0x7f52e8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100940 0x7f52e819ea40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 --2- 172.21.15.32:0/1067488094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e8101300 0x7f52e819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 >> 172.21.15.32:0/1067488094 conn(0x7f52e80fc410 msgr2=0x7f52e80fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 shutdown_connections 2023-08-11T15:45:47.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:47.765+0000 7f52ef9a8700 1 -- 172.21.15.32:0/1067488094 wait complete. 2023-08-11T15:45:47.772 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-08-11T15:45:48.210 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2023-08-11T15:45:46.090690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652882,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":253,"num_read_kb":240,"num_write":256,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337936,"kb_used_data":13688,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603888,"statfs":{"total":1151940427776,"available":1151594381312,"internally_reserved":0,"allocated":14016512,"data_stored":7824167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003371"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192439+0000","last_change":"2023-08-11T15:44:19.862927+0000","last_active":"2023-08-11T15:44:55.192439+0000","last_peered":"2023-08-11T15:44:55.192439+0000","last_clean":"2023-08-11T15:44:55.192439+0000","last_became_active":"2023-08-11T15:44:19.862626+0000","last_became_peered":"2023-08-11T15:44:19.862626+0000","last_unstale":"2023-08-11T15:44:55.192439+0000","last_undegraded":"2023-08-11T15:44:55.192439+0000","last_fullsized":"2023-08-11T15:44:55.192439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:39:26.212373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190047+0000","last_change":"2023-08-11T15:44:19.852455+0000","last_active":"2023-08-11T15:44:55.190047+0000","last_peered":"2023-08-11T15:44:55.190047+0000","last_clean":"2023-08-11T15:44:55.190047+0000","last_became_active":"2023-08-11T15:44:19.851897+0000","last_became_peered":"2023-08-11T15:44:19.851897+0000","last_unstale":"2023-08-11T15:44:55.190047+0000","last_undegraded":"2023-08-11T15:44:55.190047+0000","last_fullsized":"2023-08-11T15:44:55.190047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:23:56.473990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189740+0000","last_change":"2023-08-11T15:44:19.855232+0000","last_active":"2023-08-11T15:44:55.189740+0000","last_peered":"2023-08-11T15:44:55.189740+0000","last_clean":"2023-08-11T15:44:55.189740+0000","last_became_active":"2023-08-11T15:44:19.854940+0000","last_became_peered":"2023-08-11T15:44:19.854940+0000","last_unstale":"2023-08-11T15:44:55.189740+0000","last_undegraded":"2023-08-11T15:44:55.189740+0000","last_fullsized":"2023-08-11T15:44:55.189740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:58:15.175180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198360+0000","last_change":"2023-08-11T15:44:19.856137+0000","last_active":"2023-08-11T15:44:55.198360+0000","last_peered":"2023-08-11T15:44:55.198360+0000","last_clean":"2023-08-11T15:44:55.198360+0000","last_became_active":"2023-08-11T15:44:19.855852+0000","last_became_peered":"2023-08-11T15:44:19.855852+0000","last_unstale":"2023-08-11T15:44:55.198360+0000","last_undegraded":"2023-08-11T15:44:55.198360+0000","last_fullsized":"2023-08-11T15:44:55.198360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:50:01.697765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194593+0000","last_change":"2023-08-11T15:44:19.862618+0000","last_active":"2023-08-11T15:44:55.194593+0000","last_peered":"2023-08-11T15:44:55.194593+0000","last_clean":"2023-08-11T15:44:55.194593+0000","last_became_active":"2023-08-11T15:44:19.862265+0000","last_became_peered":"2023-08-11T15:44:19.862265+0000","last_unstale":"2023-08-11T15:44:55.194593+0000","last_undegraded":"2023-08-11T15:44:55.194593+0000","last_fullsized":"2023-08-11T15:44:55.194593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:40.449974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192805+0000","last_change":"2023-08-11T15:44:19.857801+0000","last_active":"2023-08-11T15:44:55.192805+0000","last_peered":"2023-08-11T15:44:55.192805+0000","last_clean":"2023-08-11T15:44:55.192805+0000","last_became_active":"2023-08-11T15:44:19.857388+0000","last_became_peered":"2023-08-11T15:44:19.857388+0000","last_unstale":"2023-08-11T15:44:55.192805+0000","last_undegraded":"2023-08-11T15:44:55.192805+0000","last_fullsized":"2023-08-11T15:44:55.192805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:09:13.938322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194812+0000","last_change":"2023-08-11T15:44:19.859334+0000","last_active":"2023-08-11T15:44:55.194812+0000","last_peered":"2023-08-11T15:44:55.194812+0000","last_clean":"2023-08-11T15:44:55.194812+0000","last_became_active":"2023-08-11T15:44:19.859073+0000","last_became_peered":"2023-08-11T15:44:19.859073+0000","last_unstale":"2023-08-11T15:44:55.194812+0000","last_undegraded":"2023-08-11T15:44:55.194812+0000","last_fullsized":"2023-08-11T15:44:55.194812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:40:27.803924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194562+0000","last_change":"2023-08-11T15:44:19.853064+0000","last_active":"2023-08-11T15:44:55.194562+0000","last_peered":"2023-08-11T15:44:55.194562+0000","last_clean":"2023-08-11T15:44:55.194562+0000","last_became_active":"2023-08-11T15:44:19.852736+0000","last_became_peered":"2023-08-11T15:44:19.852736+0000","last_unstale":"2023-08-11T15:44:55.194562+0000","last_undegraded":"2023-08-11T15:44:55.194562+0000","last_fullsized":"2023-08-11T15:44:55.194562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:57:13.446578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191401+0000","last_change":"2023-08-11T15:44:19.861912+0000","last_active":"2023-08-11T15:44:55.191401+0000","last_peered":"2023-08-11T15:44:55.191401+0000","last_clean":"2023-08-11T15:44:55.191401+0000","last_became_active":"2023-08-11T15:44:19.861374+0000","last_became_peered":"2023-08-11T15:44:19.861374+0000","last_unstale":"2023-08-11T15:44:55.191401+0000","last_undegraded":"2023-08-11T15:44:55.191401+0000","last_fullsized":"2023-08-11T15:44:55.191401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:43:22.890968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193523+0000","last_change":"2023-08-11T15:44:19.858862+0000","last_active":"2023-08-11T15:44:55.193523+0000","last_peered":"2023-08-11T15:44:55.193523+0000","last_clean":"2023-08-11T15:44:55.193523+0000","last_became_active":"2023-08-11T15:44:19.858512+0000","last_became_peered":"2023-08-11T15:44:19.858512+0000","last_unstale":"2023-08-11T15:44:55.193523+0000","last_undegraded":"2023-08-11T15:44:55.193523+0000","last_fullsized":"2023-08-11T15:44:55.193523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:40:43.630548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195034+0000","last_change":"2023-08-11T15:44:19.864063+0000","last_active":"2023-08-11T15:44:55.195034+0000","last_peered":"2023-08-11T15:44:55.195034+0000","last_clean":"2023-08-11T15:44:55.195034+0000","last_became_active":"2023-08-11T15:44:19.863081+0000","last_became_peered":"2023-08-11T15:44:19.863081+0000","last_unstale":"2023-08-11T15:44:55.195034+0000","last_undegraded":"2023-08-11T15:44:55.195034+0000","last_fullsized":"2023-08-11T15:44:55.195034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:58:18.278385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198186+0000","last_change":"2023-08-11T15:44:19.855347+0000","last_active":"2023-08-11T15:44:55.198186+0000","last_peered":"2023-08-11T15:44:55.198186+0000","last_clean":"2023-08-11T15:44:55.198186+0000","last_became_active":"2023-08-11T15:44:19.855025+0000","last_became_peered":"2023-08-11T15:44:19.855025+0000","last_unstale":"2023-08-11T15:44:55.198186+0000","last_undegraded":"2023-08-11T15:44:55.198186+0000","last_fullsized":"2023-08-11T15:44:55.198186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:44:49.739947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195827+0000","last_change":"2023-08-11T15:44:19.862153+0000","last_active":"2023-08-11T15:44:55.195827+0000","last_peered":"2023-08-11T15:44:55.195827+0000","last_clean":"2023-08-11T15:44:55.195827+0000","last_became_active":"2023-08-11T15:44:19.861830+0000","last_became_peered":"2023-08-11T15:44:19.861830+0000","last_unstale":"2023-08-11T15:44:55.195827+0000","last_undegraded":"2023-08-11T15:44:55.195827+0000","last_fullsized":"2023-08-11T15:44:55.195827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:16:01.026944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189564+0000","last_change":"2023-08-11T15:44:19.857184+0000","last_active":"2023-08-11T15:44:55.189564+0000","last_peered":"2023-08-11T15:44:55.189564+0000","last_clean":"2023-08-11T15:44:55.189564+0000","last_became_active":"2023-08-11T15:44:19.856898+0000","last_became_peered":"2023-08-11T15:44:19.856898+0000","last_unstale":"2023-08-11T15:44:55.189564+0000","last_undegraded":"2023-08-11T15:44:55.189564+0000","last_fullsized":"2023-08-11T15:44:55.189564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:35:04.984315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189495+0000","last_change":"2023-08-11T15:44:19.852808+0000","last_active":"2023-08-11T15:44:55.189495+0000","last_peered":"2023-08-11T15:44:55.189495+0000","last_clean":"2023-08-11T15:44:55.189495+0000","last_became_active":"2023-08-11T15:44:19.852477+0000","last_became_peered":"2023-08-11T15:44:19.852477+0000","last_unstale":"2023-08-11T15:44:55.189495+0000","last_undegraded":"2023-08-11T15:44:55.189495+0000","last_fullsized":"2023-08-11T15:44:55.189495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:15.737333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190450+0000","last_change":"2023-08-11T15:44:19.856273+0000","last_active":"2023-08-11T15:44:55.190450+0000","last_peered":"2023-08-11T15:44:55.190450+0000","last_clean":"2023-08-11T15:44:55.190450+0000","last_became_active":"2023-08-11T15:44:19.854579+0000","last_became_peered":"2023-08-11T15:44:19.854579+0000","last_unstale":"2023-08-11T15:44:55.190450+0000","last_undegraded":"2023-08-11T15:44:55.190450+0000","last_fullsized":"2023-08-11T15:44:55.190450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:02:07.058213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192239+0000","last_change":"2023-08-11T15:44:19.866820+0000","last_active":"2023-08-11T15:44:55.192239+0000","last_peered":"2023-08-11T15:44:55.192239+0000","last_clean":"2023-08-11T15:44:55.192239+0000","last_became_active":"2023-08-11T15:44:19.866507+0000","last_became_peered":"2023-08-11T15:44:19.866507+0000","last_unstale":"2023-08-11T15:44:55.192239+0000","last_undegraded":"2023-08-11T15:44:55.192239+0000","last_fullsized":"2023-08-11T15:44:55.192239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:41:54.967368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193782+0000","last_change":"2023-08-11T15:44:19.865821+0000","last_active":"2023-08-11T15:44:55.193782+0000","last_peered":"2023-08-11T15:44:55.193782+0000","last_clean":"2023-08-11T15:44:55.193782+0000","last_became_active":"2023-08-11T15:44:19.865510+0000","last_became_peered":"2023-08-11T15:44:19.865510+0000","last_unstale":"2023-08-11T15:44:55.193782+0000","last_undegraded":"2023-08-11T15:44:55.193782+0000","last_fullsized":"2023-08-11T15:44:55.193782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:20:32.474091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194714+0000","last_change":"2023-08-11T15:44:19.863079+0000","last_active":"2023-08-11T15:44:55.194714+0000","last_peered":"2023-08-11T15:44:55.194714+0000","last_clean":"2023-08-11T15:44:55.194714+0000","last_became_active":"2023-08-11T15:44:19.862663+0000","last_became_peered":"2023-08-11T15:44:19.862663+0000","last_unstale":"2023-08-11T15:44:55.194714+0000","last_undegraded":"2023-08-11T15:44:55.194714+0000","last_fullsized":"2023-08-11T15:44:55.194714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:41:51.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194903+0000","last_change":"2023-08-11T15:44:24.201308+0000","last_active":"2023-08-11T15:44:55.194903+0000","last_peered":"2023-08-11T15:44:55.194903+0000","last_clean":"2023-08-11T15:44:55.194903+0000","last_became_active":"2023-08-11T15:44:24.200884+0000","last_became_peered":"2023-08-11T15:44:24.200884+0000","last_unstale":"2023-08-11T15:44:55.194903+0000","last_undegraded":"2023-08-11T15:44:55.194903+0000","last_fullsized":"2023-08-11T15:44:55.194903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:59.419400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193546+0000","last_change":"2023-08-11T15:44:19.155621+0000","last_active":"2023-08-11T15:44:55.193546+0000","last_peered":"2023-08-11T15:44:55.193546+0000","last_clean":"2023-08-11T15:44:55.193546+0000","last_became_active":"2023-08-11T15:44:19.154494+0000","last_became_peered":"2023-08-11T15:44:19.154494+0000","last_unstale":"2023-08-11T15:44:55.193546+0000","last_undegraded":"2023-08-11T15:44:55.193546+0000","last_fullsized":"2023-08-11T15:44:55.193546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:09.243769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196857+0000","last_change":"2023-08-11T15:44:19.853101+0000","last_active":"2023-08-11T15:44:55.196857+0000","last_peered":"2023-08-11T15:44:55.196857+0000","last_clean":"2023-08-11T15:44:55.196857+0000","last_became_active":"2023-08-11T15:44:19.852804+0000","last_became_peered":"2023-08-11T15:44:19.852804+0000","last_unstale":"2023-08-11T15:44:55.196857+0000","last_undegraded":"2023-08-11T15:44:55.196857+0000","last_fullsized":"2023-08-11T15:44:55.196857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:26:49.173800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200726+0000","last_change":"2023-08-11T15:44:55.200726+0000","last_active":"2023-08-11T15:44:55.200726+0000","last_peered":"2023-08-11T15:44:55.200726+0000","last_clean":"2023-08-11T15:44:55.200726+0000","last_became_active":"2023-08-11T15:44:55.200360+0000","last_became_peered":"2023-08-11T15:44:55.200360+0000","last_unstale":"2023-08-11T15:44:55.200726+0000","last_undegraded":"2023-08-11T15:44:55.200726+0000","last_fullsized":"2023-08-11T15:44:55.200726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:59:09.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196984+0000","last_change":"2023-08-11T15:44:19.147219+0000","last_active":"2023-08-11T15:44:55.196984+0000","last_peered":"2023-08-11T15:44:55.196984+0000","last_clean":"2023-08-11T15:44:55.196984+0000","last_became_active":"2023-08-11T15:44:19.146874+0000","last_became_peered":"2023-08-11T15:44:19.146874+0000","last_unstale":"2023-08-11T15:44:55.196984+0000","last_undegraded":"2023-08-11T15:44:55.196984+0000","last_fullsized":"2023-08-11T15:44:55.196984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:33.758701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189983+0000","last_change":"2023-08-11T15:44:19.854103+0000","last_active":"2023-08-11T15:44:55.189983+0000","last_peered":"2023-08-11T15:44:55.189983+0000","last_clean":"2023-08-11T15:44:55.189983+0000","last_became_active":"2023-08-11T15:44:19.853542+0000","last_became_peered":"2023-08-11T15:44:19.853542+0000","last_unstale":"2023-08-11T15:44:55.189983+0000","last_undegraded":"2023-08-11T15:44:55.189983+0000","last_fullsized":"2023-08-11T15:44:55.189983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:35:27.247168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200630+0000","last_change":"2023-08-11T15:44:55.200630+0000","last_active":"2023-08-11T15:44:55.200630+0000","last_peered":"2023-08-11T15:44:55.200630+0000","last_clean":"2023-08-11T15:44:55.200630+0000","last_became_active":"2023-08-11T15:44:55.200271+0000","last_became_peered":"2023-08-11T15:44:55.200271+0000","last_unstale":"2023-08-11T15:44:55.200630+0000","last_undegraded":"2023-08-11T15:44:55.200630+0000","last_fullsized":"2023-08-11T15:44:55.200630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:50:50.162322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195272+0000","last_change":"2023-08-11T15:44:19.148019+0000","last_active":"2023-08-11T15:44:55.195272+0000","last_peered":"2023-08-11T15:44:55.195272+0000","last_clean":"2023-08-11T15:44:55.195272+0000","last_became_active":"2023-08-11T15:44:19.147501+0000","last_became_peered":"2023-08-11T15:44:19.147501+0000","last_unstale":"2023-08-11T15:44:55.195272+0000","last_undegraded":"2023-08-11T15:44:55.195272+0000","last_fullsized":"2023-08-11T15:44:55.195272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:32.333344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190689+0000","last_change":"2023-08-11T15:44:19.865230+0000","last_active":"2023-08-11T15:44:55.190689+0000","last_peered":"2023-08-11T15:44:55.190689+0000","last_clean":"2023-08-11T15:44:55.190689+0000","last_became_active":"2023-08-11T15:44:19.864838+0000","last_became_peered":"2023-08-11T15:44:19.864838+0000","last_unstale":"2023-08-11T15:44:55.190689+0000","last_undegraded":"2023-08-11T15:44:55.190689+0000","last_fullsized":"2023-08-11T15:44:55.190689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:10.222090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198211+0000","last_change":"2023-08-11T15:44:24.192282+0000","last_active":"2023-08-11T15:44:55.198211+0000","last_peered":"2023-08-11T15:44:55.198211+0000","last_clean":"2023-08-11T15:44:55.198211+0000","last_became_active":"2023-08-11T15:44:24.191972+0000","last_became_peered":"2023-08-11T15:44:24.191972+0000","last_unstale":"2023-08-11T15:44:55.198211+0000","last_undegraded":"2023-08-11T15:44:55.198211+0000","last_fullsized":"2023-08-11T15:44:55.198211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:14:12.777720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195406+0000","last_change":"2023-08-11T15:44:19.147181+0000","last_active":"2023-08-11T15:44:55.195406+0000","last_peered":"2023-08-11T15:44:55.195406+0000","last_clean":"2023-08-11T15:44:55.195406+0000","last_became_active":"2023-08-11T15:44:19.146843+0000","last_became_peered":"2023-08-11T15:44:19.146843+0000","last_unstale":"2023-08-11T15:44:55.195406+0000","last_undegraded":"2023-08-11T15:44:55.195406+0000","last_fullsized":"2023-08-11T15:44:55.195406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:39:50.508765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200249+0000","last_change":"2023-08-11T15:44:55.199783+0000","last_active":"2023-08-11T15:44:55.200249+0000","last_peered":"2023-08-11T15:44:55.200249+0000","last_clean":"2023-08-11T15:44:55.200249+0000","last_became_active":"2023-08-11T15:44:55.199391+0000","last_became_peered":"2023-08-11T15:44:55.199391+0000","last_unstale":"2023-08-11T15:44:55.200249+0000","last_undegraded":"2023-08-11T15:44:55.200249+0000","last_fullsized":"2023-08-11T15:44:55.200249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:19:02.132279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194561+0000","last_change":"2023-08-11T15:44:24.192621+0000","last_active":"2023-08-11T15:44:55.194561+0000","last_peered":"2023-08-11T15:44:55.194561+0000","last_clean":"2023-08-11T15:44:55.194561+0000","last_became_active":"2023-08-11T15:44:24.192183+0000","last_became_peered":"2023-08-11T15:44:24.192183+0000","last_unstale":"2023-08-11T15:44:55.194561+0000","last_undegraded":"2023-08-11T15:44:55.194561+0000","last_fullsized":"2023-08-11T15:44:55.194561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:16.822794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190900+0000","last_change":"2023-08-11T15:44:19.149437+0000","last_active":"2023-08-11T15:44:55.190900+0000","last_peered":"2023-08-11T15:44:55.190900+0000","last_clean":"2023-08-11T15:44:55.190900+0000","last_became_active":"2023-08-11T15:44:19.149086+0000","last_became_peered":"2023-08-11T15:44:19.149086+0000","last_unstale":"2023-08-11T15:44:55.190900+0000","last_undegraded":"2023-08-11T15:44:55.190900+0000","last_fullsized":"2023-08-11T15:44:55.190900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:50.716278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195215+0000","last_change":"2023-08-11T15:44:19.865452+0000","last_active":"2023-08-11T15:44:55.195215+0000","last_peered":"2023-08-11T15:44:55.195215+0000","last_clean":"2023-08-11T15:44:55.195215+0000","last_became_active":"2023-08-11T15:44:19.865133+0000","last_became_peered":"2023-08-11T15:44:19.865133+0000","last_unstale":"2023-08-11T15:44:55.195215+0000","last_undegraded":"2023-08-11T15:44:55.195215+0000","last_fullsized":"2023-08-11T15:44:55.195215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:42:54.760249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195540+0000","last_change":"2023-08-11T15:44:24.201817+0000","last_active":"2023-08-11T15:44:55.195540+0000","last_peered":"2023-08-11T15:44:55.195540+0000","last_clean":"2023-08-11T15:44:55.195540+0000","last_became_active":"2023-08-11T15:44:24.200766+0000","last_became_peered":"2023-08-11T15:44:24.200766+0000","last_unstale":"2023-08-11T15:44:55.195540+0000","last_undegraded":"2023-08-11T15:44:55.195540+0000","last_fullsized":"2023-08-11T15:44:55.195540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:46:08.346089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192629+0000","last_change":"2023-08-11T15:44:19.151213+0000","last_active":"2023-08-11T15:44:55.192629+0000","last_peered":"2023-08-11T15:44:55.192629+0000","last_clean":"2023-08-11T15:44:55.192629+0000","last_became_active":"2023-08-11T15:44:19.150914+0000","last_became_peered":"2023-08-11T15:44:19.150914+0000","last_unstale":"2023-08-11T15:44:55.192629+0000","last_undegraded":"2023-08-11T15:44:55.192629+0000","last_fullsized":"2023-08-11T15:44:55.192629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:22:30.877228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191123+0000","last_change":"2023-08-11T15:44:19.858238+0000","last_active":"2023-08-11T15:44:55.191123+0000","last_peered":"2023-08-11T15:44:55.191123+0000","last_clean":"2023-08-11T15:44:55.191123+0000","last_became_active":"2023-08-11T15:44:19.857919+0000","last_became_peered":"2023-08-11T15:44:19.857919+0000","last_unstale":"2023-08-11T15:44:55.191123+0000","last_undegraded":"2023-08-11T15:44:55.191123+0000","last_fullsized":"2023-08-11T15:44:55.191123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:15:28.098691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192085+0000","last_change":"2023-08-11T15:44:24.201485+0000","last_active":"2023-08-11T15:44:55.192085+0000","last_peered":"2023-08-11T15:44:55.192085+0000","last_clean":"2023-08-11T15:44:55.192085+0000","last_became_active":"2023-08-11T15:44:24.200984+0000","last_became_peered":"2023-08-11T15:44:24.200984+0000","last_unstale":"2023-08-11T15:44:55.192085+0000","last_undegraded":"2023-08-11T15:44:55.192085+0000","last_fullsized":"2023-08-11T15:44:55.192085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:57.566307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195775+0000","last_change":"2023-08-11T15:44:19.149289+0000","last_active":"2023-08-11T15:44:55.195775+0000","last_peered":"2023-08-11T15:44:55.195775+0000","last_clean":"2023-08-11T15:44:55.195775+0000","last_became_active":"2023-08-11T15:44:19.148881+0000","last_became_peered":"2023-08-11T15:44:19.148881+0000","last_unstale":"2023-08-11T15:44:55.195775+0000","last_undegraded":"2023-08-11T15:44:55.195775+0000","last_fullsized":"2023-08-11T15:44:55.195775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:02.163702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194495+0000","last_change":"2023-08-11T15:44:19.867860+0000","last_active":"2023-08-11T15:44:55.194495+0000","last_peered":"2023-08-11T15:44:55.194495+0000","last_clean":"2023-08-11T15:44:55.194495+0000","last_became_active":"2023-08-11T15:44:19.867545+0000","last_became_peered":"2023-08-11T15:44:19.867545+0000","last_unstale":"2023-08-11T15:44:55.194495+0000","last_undegraded":"2023-08-11T15:44:55.194495+0000","last_fullsized":"2023-08-11T15:44:55.194495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:46:29.660791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189976+0000","last_change":"2023-08-11T15:44:24.197251+0000","last_active":"2023-08-11T15:44:55.189976+0000","last_peered":"2023-08-11T15:44:55.189976+0000","last_clean":"2023-08-11T15:44:55.189976+0000","last_became_active":"2023-08-11T15:44:24.196735+0000","last_became_peered":"2023-08-11T15:44:24.196735+0000","last_unstale":"2023-08-11T15:44:55.189976+0000","last_undegraded":"2023-08-11T15:44:55.189976+0000","last_fullsized":"2023-08-11T15:44:55.189976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:04:00.057964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191849+0000","last_change":"2023-08-11T15:44:19.146196+0000","last_active":"2023-08-11T15:44:55.191849+0000","last_peered":"2023-08-11T15:44:55.191849+0000","last_clean":"2023-08-11T15:44:55.191849+0000","last_became_active":"2023-08-11T15:44:19.145812+0000","last_became_peered":"2023-08-11T15:44:19.145812+0000","last_unstale":"2023-08-11T15:44:55.191849+0000","last_undegraded":"2023-08-11T15:44:55.191849+0000","last_fullsized":"2023-08-11T15:44:55.191849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:51:59.428704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192831+0000","last_change":"2023-08-11T15:44:19.855389+0000","last_active":"2023-08-11T15:44:55.192831+0000","last_peered":"2023-08-11T15:44:55.192831+0000","last_clean":"2023-08-11T15:44:55.192831+0000","last_became_active":"2023-08-11T15:44:19.855063+0000","last_became_peered":"2023-08-11T15:44:19.855063+0000","last_unstale":"2023-08-11T15:44:55.192831+0000","last_undegraded":"2023-08-11T15:44:55.192831+0000","last_fullsized":"2023-08-11T15:44:55.192831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:42:37.301427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194782+0000","last_change":"2023-08-11T15:44:24.192450+0000","last_active":"2023-08-11T15:44:55.194782+0000","last_peered":"2023-08-11T15:44:55.194782+0000","last_clean":"2023-08-11T15:44:55.194782+0000","last_became_active":"2023-08-11T15:44:24.191989+0000","last_became_peered":"2023-08-11T15:44:24.191989+0000","last_unstale":"2023-08-11T15:44:55.194782+0000","last_undegraded":"2023-08-11T15:44:55.194782+0000","last_fullsized":"2023-08-11T15:44:55.194782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:06.182448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192857+0000","last_change":"2023-08-11T15:44:19.149002+0000","last_active":"2023-08-11T15:44:55.192857+0000","last_peered":"2023-08-11T15:44:55.192857+0000","last_clean":"2023-08-11T15:44:55.192857+0000","last_became_active":"2023-08-11T15:44:19.148635+0000","last_became_peered":"2023-08-11T15:44:19.148635+0000","last_unstale":"2023-08-11T15:44:55.192857+0000","last_undegraded":"2023-08-11T15:44:55.192857+0000","last_fullsized":"2023-08-11T15:44:55.192857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:58.450017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191583+0000","last_change":"2023-08-11T15:44:19.858067+0000","last_active":"2023-08-11T15:44:55.191583+0000","last_peered":"2023-08-11T15:44:55.191583+0000","last_clean":"2023-08-11T15:44:55.191583+0000","last_became_active":"2023-08-11T15:44:19.856641+0000","last_became_peered":"2023-08-11T15:44:19.856641+0000","last_unstale":"2023-08-11T15:44:55.191583+0000","last_undegraded":"2023-08-11T15:44:55.191583+0000","last_fullsized":"2023-08-11T15:44:55.191583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:17.780319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192698+0000","last_change":"2023-08-11T15:44:24.196806+0000","last_active":"2023-08-11T15:44:55.192698+0000","last_peered":"2023-08-11T15:44:55.192698+0000","last_clean":"2023-08-11T15:44:55.192698+0000","last_became_active":"2023-08-11T15:44:24.195268+0000","last_became_peered":"2023-08-11T15:44:24.195268+0000","last_unstale":"2023-08-11T15:44:55.192698+0000","last_undegraded":"2023-08-11T15:44:55.192698+0000","last_fullsized":"2023-08-11T15:44:55.192698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:47:43.203448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.202786+0000","last_change":"2023-08-11T15:44:55.202786+0000","last_active":"2023-08-11T15:44:55.202786+0000","last_peered":"2023-08-11T15:44:55.202786+0000","last_clean":"2023-08-11T15:44:55.202786+0000","last_became_active":"2023-08-11T15:44:55.202409+0000","last_became_peered":"2023-08-11T15:44:55.202409+0000","last_unstale":"2023-08-11T15:44:55.202786+0000","last_undegraded":"2023-08-11T15:44:55.202786+0000","last_fullsized":"2023-08-11T15:44:55.202786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:22.000673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190025+0000","last_change":"2023-08-11T15:44:19.853881+0000","last_active":"2023-08-11T15:44:55.190025+0000","last_peered":"2023-08-11T15:44:55.190025+0000","last_clean":"2023-08-11T15:44:55.190025+0000","last_became_active":"2023-08-11T15:44:19.853139+0000","last_became_peered":"2023-08-11T15:44:19.853139+0000","last_unstale":"2023-08-11T15:44:55.190025+0000","last_undegraded":"2023-08-11T15:44:55.190025+0000","last_fullsized":"2023-08-11T15:44:55.190025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:23:44.862419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197483+0000","last_change":"2023-08-11T15:44:24.201620+0000","last_active":"2023-08-11T15:44:55.197483+0000","last_peered":"2023-08-11T15:44:55.197483+0000","last_clean":"2023-08-11T15:44:55.197483+0000","last_became_active":"2023-08-11T15:44:24.201162+0000","last_became_peered":"2023-08-11T15:44:24.201162+0000","last_unstale":"2023-08-11T15:44:55.197483+0000","last_undegraded":"2023-08-11T15:44:55.197483+0000","last_fullsized":"2023-08-11T15:44:55.197483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:06:42.850458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191954+0000","last_change":"2023-08-11T15:44:19.156899+0000","last_active":"2023-08-11T15:44:55.191954+0000","last_peered":"2023-08-11T15:44:55.191954+0000","last_clean":"2023-08-11T15:44:55.191954+0000","last_became_active":"2023-08-11T15:44:19.156643+0000","last_became_peered":"2023-08-11T15:44:19.156643+0000","last_unstale":"2023-08-11T15:44:55.191954+0000","last_undegraded":"2023-08-11T15:44:55.191954+0000","last_fullsized":"2023-08-11T15:44:55.191954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:07.395481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190798+0000","last_change":"2023-08-11T15:44:19.857828+0000","last_active":"2023-08-11T15:44:55.190798+0000","last_peered":"2023-08-11T15:44:55.190798+0000","last_clean":"2023-08-11T15:44:55.190798+0000","last_became_active":"2023-08-11T15:44:19.856781+0000","last_became_peered":"2023-08-11T15:44:19.856781+0000","last_unstale":"2023-08-11T15:44:55.190798+0000","last_undegraded":"2023-08-11T15:44:55.190798+0000","last_fullsized":"2023-08-11T15:44:55.190798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:59:52.008608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192640+0000","last_change":"2023-08-11T15:44:24.193213+0000","last_active":"2023-08-11T15:44:55.192640+0000","last_peered":"2023-08-11T15:44:55.192640+0000","last_clean":"2023-08-11T15:44:55.192640+0000","last_became_active":"2023-08-11T15:44:24.192819+0000","last_became_peered":"2023-08-11T15:44:24.192819+0000","last_unstale":"2023-08-11T15:44:55.192640+0000","last_undegraded":"2023-08-11T15:44:55.192640+0000","last_fullsized":"2023-08-11T15:44:55.192640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:21:03.566682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:45:42.299277+0000","last_change":"2023-08-11T15:44:19.149753+0000","last_active":"2023-08-11T15:45:42.299277+0000","last_peered":"2023-08-11T15:45:42.299277+0000","last_clean":"2023-08-11T15:45:42.299277+0000","last_became_active":"2023-08-11T15:44:19.149420+0000","last_became_peered":"2023-08-11T15:44:19.149420+0000","last_unstale":"2023-08-11T15:45:42.299277+0000","last_undegraded":"2023-08-11T15:45:42.299277+0000","last_fullsized":"2023-08-11T15:45:42.299277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:43:16.393961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61470,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":111,"num_read_kb":115,"num_write":30,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195024+0000","last_change":"2023-08-11T15:44:19.861342+0000","last_active":"2023-08-11T15:44:55.195024+0000","last_peered":"2023-08-11T15:44:55.195024+0000","last_clean":"2023-08-11T15:44:55.195024+0000","last_became_active":"2023-08-11T15:44:19.860978+0000","last_became_peered":"2023-08-11T15:44:19.860978+0000","last_unstale":"2023-08-11T15:44:55.195024+0000","last_undegraded":"2023-08-11T15:44:55.195024+0000","last_fullsized":"2023-08-11T15:44:55.195024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:07:12.643255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191794+0000","last_change":"2023-08-11T15:44:24.191504+0000","last_active":"2023-08-11T15:44:55.191794+0000","last_peered":"2023-08-11T15:44:55.191794+0000","last_clean":"2023-08-11T15:44:55.191794+0000","last_became_active":"2023-08-11T15:44:24.191127+0000","last_became_peered":"2023-08-11T15:44:24.191127+0000","last_unstale":"2023-08-11T15:44:55.191794+0000","last_undegraded":"2023-08-11T15:44:55.191794+0000","last_fullsized":"2023-08-11T15:44:55.191794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:47:16.917614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194641+0000","last_change":"2023-08-11T15:44:19.148354+0000","last_active":"2023-08-11T15:44:55.194641+0000","last_peered":"2023-08-11T15:44:55.194641+0000","last_clean":"2023-08-11T15:44:55.194641+0000","last_became_active":"2023-08-11T15:44:19.148036+0000","last_became_peered":"2023-08-11T15:44:19.148036+0000","last_unstale":"2023-08-11T15:44:55.194641+0000","last_undegraded":"2023-08-11T15:44:55.194641+0000","last_fullsized":"2023-08-11T15:44:55.194641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:06:50.544324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195295+0000","last_change":"2023-08-11T15:44:19.862409+0000","last_active":"2023-08-11T15:44:55.195295+0000","last_peered":"2023-08-11T15:44:55.195295+0000","last_clean":"2023-08-11T15:44:55.195295+0000","last_became_active":"2023-08-11T15:44:19.861359+0000","last_became_peered":"2023-08-11T15:44:19.861359+0000","last_unstale":"2023-08-11T15:44:55.195295+0000","last_undegraded":"2023-08-11T15:44:55.195295+0000","last_fullsized":"2023-08-11T15:44:55.195295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:22:15.811283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192490+0000","last_change":"2023-08-11T15:44:24.197634+0000","last_active":"2023-08-11T15:44:55.192490+0000","last_peered":"2023-08-11T15:44:55.192490+0000","last_clean":"2023-08-11T15:44:55.192490+0000","last_became_active":"2023-08-11T15:44:24.197181+0000","last_became_peered":"2023-08-11T15:44:24.197181+0000","last_unstale":"2023-08-11T15:44:55.192490+0000","last_undegraded":"2023-08-11T15:44:55.192490+0000","last_fullsized":"2023-08-11T15:44:55.192490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:24:25.505006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190865+0000","last_change":"2023-08-11T15:44:19.147550+0000","last_active":"2023-08-11T15:44:55.190865+0000","last_peered":"2023-08-11T15:44:55.190865+0000","last_clean":"2023-08-11T15:44:55.190865+0000","last_became_active":"2023-08-11T15:44:19.147299+0000","last_became_peered":"2023-08-11T15:44:19.147299+0000","last_unstale":"2023-08-11T15:44:55.190865+0000","last_undegraded":"2023-08-11T15:44:55.190865+0000","last_fullsized":"2023-08-11T15:44:55.190865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:16.012041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194683+0000","last_change":"2023-08-11T15:44:19.862664+0000","last_active":"2023-08-11T15:44:55.194683+0000","last_peered":"2023-08-11T15:44:55.194683+0000","last_clean":"2023-08-11T15:44:55.194683+0000","last_became_active":"2023-08-11T15:44:19.862130+0000","last_became_peered":"2023-08-11T15:44:19.862130+0000","last_unstale":"2023-08-11T15:44:55.194683+0000","last_undegraded":"2023-08-11T15:44:55.194683+0000","last_fullsized":"2023-08-11T15:44:55.194683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:26:44.167213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193403+0000","last_change":"2023-08-11T15:44:24.199851+0000","last_active":"2023-08-11T15:44:55.193403+0000","last_peered":"2023-08-11T15:44:55.193403+0000","last_clean":"2023-08-11T15:44:55.193403+0000","last_became_active":"2023-08-11T15:44:24.199420+0000","last_became_peered":"2023-08-11T15:44:24.199420+0000","last_unstale":"2023-08-11T15:44:55.193403+0000","last_undegraded":"2023-08-11T15:44:55.193403+0000","last_fullsized":"2023-08-11T15:44:55.193403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:32.647518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198075+0000","last_change":"2023-08-11T15:44:19.145888+0000","last_active":"2023-08-11T15:44:55.198075+0000","last_peered":"2023-08-11T15:44:55.198075+0000","last_clean":"2023-08-11T15:44:55.198075+0000","last_became_active":"2023-08-11T15:44:19.145459+0000","last_became_peered":"2023-08-11T15:44:19.145459+0000","last_unstale":"2023-08-11T15:44:55.198075+0000","last_undegraded":"2023-08-11T15:44:55.198075+0000","last_fullsized":"2023-08-11T15:44:55.198075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:29.167474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194230+0000","last_change":"2023-08-11T15:44:19.860035+0000","last_active":"2023-08-11T15:44:55.194230+0000","last_peered":"2023-08-11T15:44:55.194230+0000","last_clean":"2023-08-11T15:44:55.194230+0000","last_became_active":"2023-08-11T15:44:19.859696+0000","last_became_peered":"2023-08-11T15:44:19.859696+0000","last_unstale":"2023-08-11T15:44:55.194230+0000","last_undegraded":"2023-08-11T15:44:55.194230+0000","last_fullsized":"2023-08-11T15:44:55.194230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:30:33.718669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196792+0000","last_change":"2023-08-11T15:44:24.198255+0000","last_active":"2023-08-11T15:44:55.196792+0000","last_peered":"2023-08-11T15:44:55.196792+0000","last_clean":"2023-08-11T15:44:55.196792+0000","last_became_active":"2023-08-11T15:44:24.197825+0000","last_became_peered":"2023-08-11T15:44:24.197825+0000","last_unstale":"2023-08-11T15:44:55.196792+0000","last_undegraded":"2023-08-11T15:44:55.196792+0000","last_fullsized":"2023-08-11T15:44:55.196792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:00:15.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.207182+0000","last_change":"2023-08-11T15:44:55.206745+0000","last_active":"2023-08-11T15:44:55.207182+0000","last_peered":"2023-08-11T15:44:55.207182+0000","last_clean":"2023-08-11T15:44:55.207182+0000","last_became_active":"2023-08-11T15:44:55.196900+0000","last_became_peered":"2023-08-11T15:44:55.196900+0000","last_unstale":"2023-08-11T15:44:55.207182+0000","last_undegraded":"2023-08-11T15:44:55.207182+0000","last_fullsized":"2023-08-11T15:44:55.207182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:32.529218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190529+0000","last_change":"2023-08-11T15:44:19.853273+0000","last_active":"2023-08-11T15:44:55.190529+0000","last_peered":"2023-08-11T15:44:55.190529+0000","last_clean":"2023-08-11T15:44:55.190529+0000","last_became_active":"2023-08-11T15:44:19.852914+0000","last_became_peered":"2023-08-11T15:44:19.852914+0000","last_unstale":"2023-08-11T15:44:55.190529+0000","last_undegraded":"2023-08-11T15:44:55.190529+0000","last_fullsized":"2023-08-11T15:44:55.190529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:28:33.263048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190457+0000","last_change":"2023-08-11T15:44:24.194429+0000","last_active":"2023-08-11T15:44:55.190457+0000","last_peered":"2023-08-11T15:44:55.190457+0000","last_clean":"2023-08-11T15:44:55.190457+0000","last_became_active":"2023-08-11T15:44:24.194089+0000","last_became_peered":"2023-08-11T15:44:24.194089+0000","last_unstale":"2023-08-11T15:44:55.190457+0000","last_undegraded":"2023-08-11T15:44:55.190457+0000","last_fullsized":"2023-08-11T15:44:55.190457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:20:34.396873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195649+0000","last_change":"2023-08-11T15:44:19.151072+0000","last_active":"2023-08-11T15:44:55.195649+0000","last_peered":"2023-08-11T15:44:55.195649+0000","last_clean":"2023-08-11T15:44:55.195649+0000","last_became_active":"2023-08-11T15:44:19.147713+0000","last_became_peered":"2023-08-11T15:44:19.147713+0000","last_unstale":"2023-08-11T15:44:55.195649+0000","last_undegraded":"2023-08-11T15:44:55.195649+0000","last_fullsized":"2023-08-11T15:44:55.195649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:43.361011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192121+0000","last_change":"2023-08-11T15:44:19.862259+0000","last_active":"2023-08-11T15:44:55.192121+0000","last_peered":"2023-08-11T15:44:55.192121+0000","last_clean":"2023-08-11T15:44:55.192121+0000","last_became_active":"2023-08-11T15:44:19.861910+0000","last_became_peered":"2023-08-11T15:44:19.861910+0000","last_unstale":"2023-08-11T15:44:55.192121+0000","last_undegraded":"2023-08-11T15:44:55.192121+0000","last_fullsized":"2023-08-11T15:44:55.192121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:31.107437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198879+0000","last_change":"2023-08-11T15:44:55.198879+0000","last_active":"2023-08-11T15:44:55.198879+0000","last_peered":"2023-08-11T15:44:55.198879+0000","last_clean":"2023-08-11T15:44:55.198879+0000","last_became_active":"2023-08-11T15:44:55.198461+0000","last_became_peered":"2023-08-11T15:44:55.198461+0000","last_unstale":"2023-08-11T15:44:55.198879+0000","last_undegraded":"2023-08-11T15:44:55.198879+0000","last_fullsized":"2023-08-11T15:44:55.198879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:04:45.789628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193224+0000","last_change":"2023-08-11T15:44:19.152608+0000","last_active":"2023-08-11T15:44:55.193224+0000","last_peered":"2023-08-11T15:44:55.193224+0000","last_clean":"2023-08-11T15:44:55.193224+0000","last_became_active":"2023-08-11T15:44:19.152255+0000","last_became_peered":"2023-08-11T15:44:19.152255+0000","last_unstale":"2023-08-11T15:44:55.193224+0000","last_undegraded":"2023-08-11T15:44:55.193224+0000","last_fullsized":"2023-08-11T15:44:55.193224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:30:01.248200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193585+0000","last_change":"2023-08-11T15:44:19.864003+0000","last_active":"2023-08-11T15:44:55.193585+0000","last_peered":"2023-08-11T15:44:55.193585+0000","last_clean":"2023-08-11T15:44:55.193585+0000","last_became_active":"2023-08-11T15:44:19.863661+0000","last_became_peered":"2023-08-11T15:44:19.863661+0000","last_unstale":"2023-08-11T15:44:55.193585+0000","last_undegraded":"2023-08-11T15:44:55.193585+0000","last_fullsized":"2023-08-11T15:44:55.193585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:40:30.368843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196098+0000","last_change":"2023-08-11T15:44:24.194123+0000","last_active":"2023-08-11T15:44:55.196098+0000","last_peered":"2023-08-11T15:44:55.196098+0000","last_clean":"2023-08-11T15:44:55.196098+0000","last_became_active":"2023-08-11T15:44:24.193729+0000","last_became_peered":"2023-08-11T15:44:24.193729+0000","last_unstale":"2023-08-11T15:44:55.196098+0000","last_undegraded":"2023-08-11T15:44:55.196098+0000","last_fullsized":"2023-08-11T15:44:55.196098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:46:01.331338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194389+0000","last_change":"2023-08-11T15:44:19.151367+0000","last_active":"2023-08-11T15:44:55.194389+0000","last_peered":"2023-08-11T15:44:55.194389+0000","last_clean":"2023-08-11T15:44:55.194389+0000","last_became_active":"2023-08-11T15:44:19.150991+0000","last_became_peered":"2023-08-11T15:44:19.150991+0000","last_unstale":"2023-08-11T15:44:55.194389+0000","last_undegraded":"2023-08-11T15:44:55.194389+0000","last_fullsized":"2023-08-11T15:44:55.194389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:26:13.610294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192907+0000","last_change":"2023-08-11T15:44:19.858579+0000","last_active":"2023-08-11T15:44:55.192907+0000","last_peered":"2023-08-11T15:44:55.192907+0000","last_clean":"2023-08-11T15:44:55.192907+0000","last_became_active":"2023-08-11T15:44:19.858184+0000","last_became_peered":"2023-08-11T15:44:19.858184+0000","last_unstale":"2023-08-11T15:44:55.192907+0000","last_undegraded":"2023-08-11T15:44:55.192907+0000","last_fullsized":"2023-08-11T15:44:55.192907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:51:28.014074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196948+0000","last_change":"2023-08-11T15:44:24.300526+0000","last_active":"2023-08-11T15:44:55.196948+0000","last_peered":"2023-08-11T15:44:55.196948+0000","last_clean":"2023-08-11T15:44:55.196948+0000","last_became_active":"2023-08-11T15:44:24.299883+0000","last_became_peered":"2023-08-11T15:44:24.299883+0000","last_unstale":"2023-08-11T15:44:55.196948+0000","last_undegraded":"2023-08-11T15:44:55.196948+0000","last_fullsized":"2023-08-11T15:44:55.196948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:10.792495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191233+0000","last_change":"2023-08-11T15:44:19.858116+0000","last_active":"2023-08-11T15:44:55.191233+0000","last_peered":"2023-08-11T15:44:55.191233+0000","last_clean":"2023-08-11T15:44:55.191233+0000","last_became_active":"2023-08-11T15:44:19.857728+0000","last_became_peered":"2023-08-11T15:44:19.857728+0000","last_unstale":"2023-08-11T15:44:55.191233+0000","last_undegraded":"2023-08-11T15:44:55.191233+0000","last_fullsized":"2023-08-11T15:44:55.191233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:02:50.561098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195613+0000","last_change":"2023-08-11T15:44:19.157904+0000","last_active":"2023-08-11T15:44:55.195613+0000","last_peered":"2023-08-11T15:44:55.195613+0000","last_clean":"2023-08-11T15:44:55.195613+0000","last_became_active":"2023-08-11T15:44:19.157497+0000","last_became_peered":"2023-08-11T15:44:19.157497+0000","last_unstale":"2023-08-11T15:44:55.195613+0000","last_undegraded":"2023-08-11T15:44:55.195613+0000","last_fullsized":"2023-08-11T15:44:55.195613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:10:52.538491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194951+0000","last_change":"2023-08-11T15:44:24.201246+0000","last_active":"2023-08-11T15:44:55.194951+0000","last_peered":"2023-08-11T15:44:55.194951+0000","last_clean":"2023-08-11T15:44:55.194951+0000","last_became_active":"2023-08-11T15:44:24.200810+0000","last_became_peered":"2023-08-11T15:44:24.200810+0000","last_unstale":"2023-08-11T15:44:55.194951+0000","last_undegraded":"2023-08-11T15:44:55.194951+0000","last_fullsized":"2023-08-11T15:44:55.194951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:41:26.333082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195176+0000","last_change":"2023-08-11T15:44:19.846296+0000","last_active":"2023-08-11T15:44:55.195176+0000","last_peered":"2023-08-11T15:44:55.195176+0000","last_clean":"2023-08-11T15:44:55.195176+0000","last_became_active":"2023-08-11T15:44:19.845890+0000","last_became_peered":"2023-08-11T15:44:19.845890+0000","last_unstale":"2023-08-11T15:44:55.195176+0000","last_undegraded":"2023-08-11T15:44:55.195176+0000","last_fullsized":"2023-08-11T15:44:55.195176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:34:40.965829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197297+0000","last_change":"2023-08-11T15:44:19.144202+0000","last_active":"2023-08-11T15:44:55.197297+0000","last_peered":"2023-08-11T15:44:55.197297+0000","last_clean":"2023-08-11T15:44:55.197297+0000","last_became_active":"2023-08-11T15:44:19.143789+0000","last_became_peered":"2023-08-11T15:44:19.143789+0000","last_unstale":"2023-08-11T15:44:55.197297+0000","last_undegraded":"2023-08-11T15:44:55.197297+0000","last_fullsized":"2023-08-11T15:44:55.197297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:15.141815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193739+0000","last_change":"2023-08-11T15:43:14.593634+0000","last_active":"2023-08-11T15:44:55.193739+0000","last_peered":"2023-08-11T15:44:55.193739+0000","last_clean":"2023-08-11T15:44:55.193739+0000","last_became_active":"2023-08-11T15:43:14.593164+0000","last_became_peered":"2023-08-11T15:43:14.593164+0000","last_unstale":"2023-08-11T15:44:55.193739+0000","last_undegraded":"2023-08-11T15:44:55.193739+0000","last_fullsized":"2023-08-11T15:44:55.193739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:40:09.009169+0000","last_clean_scrub_stamp":"2023-08-11T15:40:09.009169+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:13:50.737842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190789+0000","last_change":"2023-08-11T15:44:24.300703+0000","last_active":"2023-08-11T15:44:55.190789+0000","last_peered":"2023-08-11T15:44:55.190789+0000","last_clean":"2023-08-11T15:44:55.190789+0000","last_became_active":"2023-08-11T15:44:24.300309+0000","last_became_peered":"2023-08-11T15:44:24.300309+0000","last_unstale":"2023-08-11T15:44:55.190789+0000","last_undegraded":"2023-08-11T15:44:55.190789+0000","last_fullsized":"2023-08-11T15:44:55.190789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:55:11.965214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201619+0000","last_change":"2023-08-11T15:44:55.201619+0000","last_active":"2023-08-11T15:44:55.201619+0000","last_peered":"2023-08-11T15:44:55.201619+0000","last_clean":"2023-08-11T15:44:55.201619+0000","last_became_active":"2023-08-11T15:44:55.201233+0000","last_became_peered":"2023-08-11T15:44:55.201233+0000","last_unstale":"2023-08-11T15:44:55.201619+0000","last_undegraded":"2023-08-11T15:44:55.201619+0000","last_fullsized":"2023-08-11T15:44:55.201619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:11:25.230480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193945+0000","last_change":"2023-08-11T15:44:19.148113+0000","last_active":"2023-08-11T15:44:55.193945+0000","last_peered":"2023-08-11T15:44:55.193945+0000","last_clean":"2023-08-11T15:44:55.193945+0000","last_became_active":"2023-08-11T15:44:19.147793+0000","last_became_peered":"2023-08-11T15:44:19.147793+0000","last_unstale":"2023-08-11T15:44:55.193945+0000","last_undegraded":"2023-08-11T15:44:55.193945+0000","last_fullsized":"2023-08-11T15:44:55.193945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:28.804554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191891+0000","last_change":"2023-08-11T15:44:24.194418+0000","last_active":"2023-08-11T15:44:55.191891+0000","last_peered":"2023-08-11T15:44:55.191891+0000","last_clean":"2023-08-11T15:44:55.191891+0000","last_became_active":"2023-08-11T15:44:24.193943+0000","last_became_peered":"2023-08-11T15:44:24.193943+0000","last_unstale":"2023-08-11T15:44:55.191891+0000","last_undegraded":"2023-08-11T15:44:55.191891+0000","last_fullsized":"2023-08-11T15:44:55.191891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:56:38.790282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195211+0000","last_change":"2023-08-11T15:44:19.151476+0000","last_active":"2023-08-11T15:44:55.195211+0000","last_peered":"2023-08-11T15:44:55.195211+0000","last_clean":"2023-08-11T15:44:55.195211+0000","last_became_active":"2023-08-11T15:44:19.151140+0000","last_became_peered":"2023-08-11T15:44:19.151140+0000","last_unstale":"2023-08-11T15:44:55.195211+0000","last_undegraded":"2023-08-11T15:44:55.195211+0000","last_fullsized":"2023-08-11T15:44:55.195211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:43.957395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196271+0000","last_change":"2023-08-11T15:44:19.862255+0000","last_active":"2023-08-11T15:44:55.196271+0000","last_peered":"2023-08-11T15:44:55.196271+0000","last_clean":"2023-08-11T15:44:55.196271+0000","last_became_active":"2023-08-11T15:44:19.861907+0000","last_became_peered":"2023-08-11T15:44:19.861907+0000","last_unstale":"2023-08-11T15:44:55.196271+0000","last_undegraded":"2023-08-11T15:44:55.196271+0000","last_fullsized":"2023-08-11T15:44:55.196271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:33:45.120706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194419+0000","last_change":"2023-08-11T15:44:24.300513+0000","last_active":"2023-08-11T15:44:55.194419+0000","last_peered":"2023-08-11T15:44:55.194419+0000","last_clean":"2023-08-11T15:44:55.194419+0000","last_became_active":"2023-08-11T15:44:24.300074+0000","last_became_peered":"2023-08-11T15:44:24.300074+0000","last_unstale":"2023-08-11T15:44:55.194419+0000","last_undegraded":"2023-08-11T15:44:55.194419+0000","last_fullsized":"2023-08-11T15:44:55.194419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:22:22.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:56.969234+0000","last_change":"2023-08-11T15:44:19.153794+0000","last_active":"2023-08-11T15:44:56.969234+0000","last_peered":"2023-08-11T15:44:56.969234+0000","last_clean":"2023-08-11T15:44:56.969234+0000","last_became_active":"2023-08-11T15:44:19.153392+0000","last_became_peered":"2023-08-11T15:44:19.153392+0000","last_unstale":"2023-08-11T15:44:56.969234+0000","last_undegraded":"2023-08-11T15:44:56.969234+0000","last_fullsized":"2023-08-11T15:44:56.969234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:59:18.150146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197135+0000","last_change":"2023-08-11T15:44:19.859799+0000","last_active":"2023-08-11T15:44:55.197135+0000","last_peered":"2023-08-11T15:44:55.197135+0000","last_clean":"2023-08-11T15:44:55.197135+0000","last_became_active":"2023-08-11T15:44:19.859411+0000","last_became_peered":"2023-08-11T15:44:19.859411+0000","last_unstale":"2023-08-11T15:44:55.197135+0000","last_undegraded":"2023-08-11T15:44:55.197135+0000","last_fullsized":"2023-08-11T15:44:55.197135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:50:04.069525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194744+0000","last_change":"2023-08-11T15:44:24.190272+0000","last_active":"2023-08-11T15:44:55.194744+0000","last_peered":"2023-08-11T15:44:55.194744+0000","last_clean":"2023-08-11T15:44:55.194744+0000","last_became_active":"2023-08-11T15:44:24.189849+0000","last_became_peered":"2023-08-11T15:44:24.189849+0000","last_unstale":"2023-08-11T15:44:55.194744+0000","last_undegraded":"2023-08-11T15:44:55.194744+0000","last_fullsized":"2023-08-11T15:44:55.194744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:21:45.389663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.198407+0000","last_change":"2023-08-11T15:44:19.154985+0000","last_active":"2023-08-11T15:44:55.198407+0000","last_peered":"2023-08-11T15:44:55.198407+0000","last_clean":"2023-08-11T15:44:55.198407+0000","last_became_active":"2023-08-11T15:44:19.154694+0000","last_became_peered":"2023-08-11T15:44:19.154694+0000","last_unstale":"2023-08-11T15:44:55.198407+0000","last_undegraded":"2023-08-11T15:44:55.198407+0000","last_fullsized":"2023-08-11T15:44:55.198407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:22:34.297160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190129+0000","last_change":"2023-08-11T15:44:19.852247+0000","last_active":"2023-08-11T15:44:55.190129+0000","last_peered":"2023-08-11T15:44:55.190129+0000","last_clean":"2023-08-11T15:44:55.190129+0000","last_became_active":"2023-08-11T15:44:19.851456+0000","last_became_peered":"2023-08-11T15:44:19.851456+0000","last_unstale":"2023-08-11T15:44:55.190129+0000","last_undegraded":"2023-08-11T15:44:55.190129+0000","last_fullsized":"2023-08-11T15:44:55.190129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:43:25.512545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.199076+0000","last_change":"2023-08-11T15:44:24.300440+0000","last_active":"2023-08-11T15:44:55.199076+0000","last_peered":"2023-08-11T15:44:55.199076+0000","last_clean":"2023-08-11T15:44:55.199076+0000","last_became_active":"2023-08-11T15:44:24.299981+0000","last_became_peered":"2023-08-11T15:44:24.299981+0000","last_unstale":"2023-08-11T15:44:55.199076+0000","last_undegraded":"2023-08-11T15:44:55.199076+0000","last_fullsized":"2023-08-11T15:44:55.199076+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:10:48.146770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190574+0000","last_change":"2023-08-11T15:44:19.151015+0000","last_active":"2023-08-11T15:44:55.190574+0000","last_peered":"2023-08-11T15:44:55.190574+0000","last_clean":"2023-08-11T15:44:55.190574+0000","last_became_active":"2023-08-11T15:44:19.150677+0000","last_became_peered":"2023-08-11T15:44:19.150677+0000","last_unstale":"2023-08-11T15:44:55.190574+0000","last_undegraded":"2023-08-11T15:44:55.190574+0000","last_fullsized":"2023-08-11T15:44:55.190574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:44.956238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194239+0000","last_change":"2023-08-11T15:44:19.861033+0000","last_active":"2023-08-11T15:44:55.194239+0000","last_peered":"2023-08-11T15:44:55.194239+0000","last_clean":"2023-08-11T15:44:55.194239+0000","last_became_active":"2023-08-11T15:44:19.860643+0000","last_became_peered":"2023-08-11T15:44:19.860643+0000","last_unstale":"2023-08-11T15:44:55.194239+0000","last_undegraded":"2023-08-11T15:44:55.194239+0000","last_fullsized":"2023-08-11T15:44:55.194239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:32:25.731474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.193496+0000","last_change":"2023-08-11T15:44:24.199389+0000","last_active":"2023-08-11T15:44:55.193496+0000","last_peered":"2023-08-11T15:44:55.193496+0000","last_clean":"2023-08-11T15:44:55.193496+0000","last_became_active":"2023-08-11T15:44:24.196226+0000","last_became_peered":"2023-08-11T15:44:24.196226+0000","last_unstale":"2023-08-11T15:44:55.193496+0000","last_undegraded":"2023-08-11T15:44:55.193496+0000","last_fullsized":"2023-08-11T15:44:55.193496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:05:34.885169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194914+0000","last_change":"2023-08-11T15:44:19.152895+0000","last_active":"2023-08-11T15:44:55.194914+0000","last_peered":"2023-08-11T15:44:55.194914+0000","last_clean":"2023-08-11T15:44:55.194914+0000","last_became_active":"2023-08-11T15:44:19.152577+0000","last_became_peered":"2023-08-11T15:44:19.152577+0000","last_unstale":"2023-08-11T15:44:55.194914+0000","last_undegraded":"2023-08-11T15:44:55.194914+0000","last_fullsized":"2023-08-11T15:44:55.194914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:23:00.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192632+0000","last_change":"2023-08-11T15:44:19.861101+0000","last_active":"2023-08-11T15:44:55.192632+0000","last_peered":"2023-08-11T15:44:55.192632+0000","last_clean":"2023-08-11T15:44:55.192632+0000","last_became_active":"2023-08-11T15:44:19.860529+0000","last_became_peered":"2023-08-11T15:44:19.860529+0000","last_unstale":"2023-08-11T15:44:55.192632+0000","last_undegraded":"2023-08-11T15:44:55.192632+0000","last_fullsized":"2023-08-11T15:44:55.192632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:30:49.161446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197970+0000","last_change":"2023-08-11T15:44:24.202856+0000","last_active":"2023-08-11T15:44:55.197970+0000","last_peered":"2023-08-11T15:44:55.197970+0000","last_clean":"2023-08-11T15:44:55.197970+0000","last_became_active":"2023-08-11T15:44:24.202317+0000","last_became_peered":"2023-08-11T15:44:24.202317+0000","last_unstale":"2023-08-11T15:44:55.197970+0000","last_undegraded":"2023-08-11T15:44:55.197970+0000","last_fullsized":"2023-08-11T15:44:55.197970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:41:10.081275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195594+0000","last_change":"2023-08-11T15:44:19.148711+0000","last_active":"2023-08-11T15:44:55.195594+0000","last_peered":"2023-08-11T15:44:55.195594+0000","last_clean":"2023-08-11T15:44:55.195594+0000","last_became_active":"2023-08-11T15:44:19.148361+0000","last_became_peered":"2023-08-11T15:44:19.148361+0000","last_unstale":"2023-08-11T15:44:55.195594+0000","last_undegraded":"2023-08-11T15:44:55.195594+0000","last_fullsized":"2023-08-11T15:44:55.195594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:05:10.590173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195941+0000","last_change":"2023-08-11T15:44:19.859562+0000","last_active":"2023-08-11T15:44:55.195941+0000","last_peered":"2023-08-11T15:44:55.195941+0000","last_clean":"2023-08-11T15:44:55.195941+0000","last_became_active":"2023-08-11T15:44:19.859243+0000","last_became_peered":"2023-08-11T15:44:19.859243+0000","last_unstale":"2023-08-11T15:44:55.195941+0000","last_undegraded":"2023-08-11T15:44:55.195941+0000","last_fullsized":"2023-08-11T15:44:55.195941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:50:37.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.197923+0000","last_change":"2023-08-11T15:44:24.300317+0000","last_active":"2023-08-11T15:44:55.197923+0000","last_peered":"2023-08-11T15:44:55.197923+0000","last_clean":"2023-08-11T15:44:55.197923+0000","last_became_active":"2023-08-11T15:44:24.299841+0000","last_became_peered":"2023-08-11T15:44:24.299841+0000","last_unstale":"2023-08-11T15:44:55.197923+0000","last_undegraded":"2023-08-11T15:44:55.197923+0000","last_fullsized":"2023-08-11T15:44:55.197923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:57:39.799187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194316+0000","last_change":"2023-08-11T15:44:19.149111+0000","last_active":"2023-08-11T15:44:55.194316+0000","last_peered":"2023-08-11T15:44:55.194316+0000","last_clean":"2023-08-11T15:44:55.194316+0000","last_became_active":"2023-08-11T15:44:19.148173+0000","last_became_peered":"2023-08-11T15:44:19.148173+0000","last_unstale":"2023-08-11T15:44:55.194316+0000","last_undegraded":"2023-08-11T15:44:55.194316+0000","last_fullsized":"2023-08-11T15:44:55.194316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:13:23.513921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194513+0000","last_change":"2023-08-11T15:44:19.863724+0000","last_active":"2023-08-11T15:44:55.194513+0000","last_peered":"2023-08-11T15:44:55.194513+0000","last_clean":"2023-08-11T15:44:55.194513+0000","last_became_active":"2023-08-11T15:44:19.863268+0000","last_became_peered":"2023-08-11T15:44:19.863268+0000","last_unstale":"2023-08-11T15:44:55.194513+0000","last_undegraded":"2023-08-11T15:44:55.194513+0000","last_fullsized":"2023-08-11T15:44:55.194513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:59.448457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190468+0000","last_change":"2023-08-11T15:44:24.191301+0000","last_active":"2023-08-11T15:44:55.190468+0000","last_peered":"2023-08-11T15:44:55.190468+0000","last_clean":"2023-08-11T15:44:55.190468+0000","last_became_active":"2023-08-11T15:44:24.190686+0000","last_became_peered":"2023-08-11T15:44:24.190686+0000","last_unstale":"2023-08-11T15:44:55.190468+0000","last_undegraded":"2023-08-11T15:44:55.190468+0000","last_fullsized":"2023-08-11T15:44:55.190468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:26:07.780061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195770+0000","last_change":"2023-08-11T15:44:19.147756+0000","last_active":"2023-08-11T15:44:55.195770+0000","last_peered":"2023-08-11T15:44:55.195770+0000","last_clean":"2023-08-11T15:44:55.195770+0000","last_became_active":"2023-08-11T15:44:19.147426+0000","last_became_peered":"2023-08-11T15:44:19.147426+0000","last_unstale":"2023-08-11T15:44:55.195770+0000","last_undegraded":"2023-08-11T15:44:55.195770+0000","last_fullsized":"2023-08-11T15:44:55.195770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:17:56.074888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195206+0000","last_change":"2023-08-11T15:44:19.863354+0000","last_active":"2023-08-11T15:44:55.195206+0000","last_peered":"2023-08-11T15:44:55.195206+0000","last_clean":"2023-08-11T15:44:55.195206+0000","last_became_active":"2023-08-11T15:44:19.862700+0000","last_became_peered":"2023-08-11T15:44:19.862700+0000","last_unstale":"2023-08-11T15:44:55.195206+0000","last_undegraded":"2023-08-11T15:44:55.195206+0000","last_fullsized":"2023-08-11T15:44:55.195206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:17:30.462182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189633+0000","last_change":"2023-08-11T15:44:24.204657+0000","last_active":"2023-08-11T15:44:55.189633+0000","last_peered":"2023-08-11T15:44:55.189633+0000","last_clean":"2023-08-11T15:44:55.189633+0000","last_became_active":"2023-08-11T15:44:24.204108+0000","last_became_peered":"2023-08-11T15:44:24.204108+0000","last_unstale":"2023-08-11T15:44:55.189633+0000","last_undegraded":"2023-08-11T15:44:55.189633+0000","last_fullsized":"2023-08-11T15:44:55.189633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:25.563864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.191508+0000","last_change":"2023-08-11T15:44:19.147014+0000","last_active":"2023-08-11T15:44:55.191508+0000","last_peered":"2023-08-11T15:44:55.191508+0000","last_clean":"2023-08-11T15:44:55.191508+0000","last_became_active":"2023-08-11T15:44:19.146720+0000","last_became_peered":"2023-08-11T15:44:19.146720+0000","last_unstale":"2023-08-11T15:44:55.191508+0000","last_undegraded":"2023-08-11T15:44:55.191508+0000","last_fullsized":"2023-08-11T15:44:55.191508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:44:30.278828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201693+0000","last_change":"2023-08-11T15:44:55.201693+0000","last_active":"2023-08-11T15:44:55.201693+0000","last_peered":"2023-08-11T15:44:55.201693+0000","last_clean":"2023-08-11T15:44:55.201693+0000","last_became_active":"2023-08-11T15:44:55.201360+0000","last_became_peered":"2023-08-11T15:44:55.201360+0000","last_unstale":"2023-08-11T15:44:55.201693+0000","last_undegraded":"2023-08-11T15:44:55.201693+0000","last_fullsized":"2023-08-11T15:44:55.201693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:54:43.667678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.200415+0000","last_change":"2023-08-11T15:44:55.197220+0000","last_active":"2023-08-11T15:44:55.200415+0000","last_peered":"2023-08-11T15:44:55.200415+0000","last_clean":"2023-08-11T15:44:55.200415+0000","last_became_active":"2023-08-11T15:44:55.196808+0000","last_became_peered":"2023-08-11T15:44:55.196808+0000","last_unstale":"2023-08-11T15:44:55.200415+0000","last_undegraded":"2023-08-11T15:44:55.200415+0000","last_fullsized":"2023-08-11T15:44:55.200415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:23.170471+0000","last_clean_scrub_stamp":"2023-08-11T15:44:23.170471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:42:42.724543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196765+0000","last_change":"2023-08-11T15:44:19.158301+0000","last_active":"2023-08-11T15:44:55.196765+0000","last_peered":"2023-08-11T15:44:55.196765+0000","last_clean":"2023-08-11T15:44:55.196765+0000","last_became_active":"2023-08-11T15:44:19.157990+0000","last_became_peered":"2023-08-11T15:44:19.157990+0000","last_unstale":"2023-08-11T15:44:55.196765+0000","last_undegraded":"2023-08-11T15:44:55.196765+0000","last_fullsized":"2023-08-11T15:44:55.196765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:18.081545+0000","last_clean_scrub_stamp":"2023-08-11T15:44:18.081545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:47:15.846659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195216+0000","last_change":"2023-08-11T15:44:19.857357+0000","last_active":"2023-08-11T15:44:55.195216+0000","last_peered":"2023-08-11T15:44:55.195216+0000","last_clean":"2023-08-11T15:44:55.195216+0000","last_became_active":"2023-08-11T15:44:19.857020+0000","last_became_peered":"2023-08-11T15:44:19.857020+0000","last_unstale":"2023-08-11T15:44:55.195216+0000","last_undegraded":"2023-08-11T15:44:55.195216+0000","last_fullsized":"2023-08-11T15:44:55.195216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:09:00.529783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192767+0000","last_change":"2023-08-11T15:44:19.863314+0000","last_active":"2023-08-11T15:44:55.192767+0000","last_peered":"2023-08-11T15:44:55.192767+0000","last_clean":"2023-08-11T15:44:55.192767+0000","last_became_active":"2023-08-11T15:44:19.862964+0000","last_became_peered":"2023-08-11T15:44:19.862964+0000","last_unstale":"2023-08-11T15:44:55.192767+0000","last_undegraded":"2023-08-11T15:44:55.192767+0000","last_fullsized":"2023-08-11T15:44:55.192767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:45:00.154066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195297+0000","last_change":"2023-08-11T15:44:19.850575+0000","last_active":"2023-08-11T15:44:55.195297+0000","last_peered":"2023-08-11T15:44:55.195297+0000","last_clean":"2023-08-11T15:44:55.195297+0000","last_became_active":"2023-08-11T15:44:19.850231+0000","last_became_peered":"2023-08-11T15:44:19.850231+0000","last_unstale":"2023-08-11T15:44:55.195297+0000","last_undegraded":"2023-08-11T15:44:55.195297+0000","last_fullsized":"2023-08-11T15:44:55.195297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:31:37.529897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195464+0000","last_change":"2023-08-11T15:44:19.857387+0000","last_active":"2023-08-11T15:44:55.195464+0000","last_peered":"2023-08-11T15:44:55.195464+0000","last_clean":"2023-08-11T15:44:55.195464+0000","last_became_active":"2023-08-11T15:44:19.856830+0000","last_became_peered":"2023-08-11T15:44:19.856830+0000","last_unstale":"2023-08-11T15:44:55.195464+0000","last_undegraded":"2023-08-11T15:44:55.195464+0000","last_fullsized":"2023-08-11T15:44:55.195464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:18:48.525612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.189975+0000","last_change":"2023-08-11T15:44:19.868112+0000","last_active":"2023-08-11T15:44:55.189975+0000","last_peered":"2023-08-11T15:44:55.189975+0000","last_clean":"2023-08-11T15:44:55.189975+0000","last_became_active":"2023-08-11T15:44:19.867610+0000","last_became_peered":"2023-08-11T15:44:19.867610+0000","last_unstale":"2023-08-11T15:44:55.189975+0000","last_undegraded":"2023-08-11T15:44:55.189975+0000","last_fullsized":"2023-08-11T15:44:55.189975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:13:03.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.196422+0000","last_change":"2023-08-11T15:44:19.858349+0000","last_active":"2023-08-11T15:44:55.196422+0000","last_peered":"2023-08-11T15:44:55.196422+0000","last_clean":"2023-08-11T15:44:55.196422+0000","last_became_active":"2023-08-11T15:44:19.858041+0000","last_became_peered":"2023-08-11T15:44:19.858041+0000","last_unstale":"2023-08-11T15:44:55.196422+0000","last_undegraded":"2023-08-11T15:44:55.196422+0000","last_fullsized":"2023-08-11T15:44:55.196422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:52.727062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192538+0000","last_change":"2023-08-11T15:44:19.865383+0000","last_active":"2023-08-11T15:44:55.192538+0000","last_peered":"2023-08-11T15:44:55.192538+0000","last_clean":"2023-08-11T15:44:55.192538+0000","last_became_active":"2023-08-11T15:44:19.865065+0000","last_became_peered":"2023-08-11T15:44:19.865065+0000","last_unstale":"2023-08-11T15:44:55.192538+0000","last_undegraded":"2023-08-11T15:44:55.192538+0000","last_fullsized":"2023-08-11T15:44:55.192538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:28:38.454441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195185+0000","last_change":"2023-08-11T15:44:19.859297+0000","last_active":"2023-08-11T15:44:55.195185+0000","last_peered":"2023-08-11T15:44:55.195185+0000","last_clean":"2023-08-11T15:44:55.195185+0000","last_became_active":"2023-08-11T15:44:19.857517+0000","last_became_peered":"2023-08-11T15:44:19.857517+0000","last_unstale":"2023-08-11T15:44:55.195185+0000","last_undegraded":"2023-08-11T15:44:55.195185+0000","last_fullsized":"2023-08-11T15:44:55.195185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:03:57.202219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.201849+0000","last_change":"2023-08-11T15:44:55.201849+0000","last_active":"2023-08-11T15:44:55.201849+0000","last_peered":"2023-08-11T15:44:55.201849+0000","last_clean":"2023-08-11T15:44:55.201849+0000","last_became_active":"2023-08-11T15:44:55.201480+0000","last_became_peered":"2023-08-11T15:44:55.201480+0000","last_unstale":"2023-08-11T15:44:55.201849+0000","last_undegraded":"2023-08-11T15:44:55.201849+0000","last_fullsized":"2023-08-11T15:44:55.201849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:57.682087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190486+0000","last_change":"2023-08-11T15:44:19.861953+0000","last_active":"2023-08-11T15:44:55.190486+0000","last_peered":"2023-08-11T15:44:55.190486+0000","last_clean":"2023-08-11T15:44:55.190486+0000","last_became_active":"2023-08-11T15:44:19.861523+0000","last_became_peered":"2023-08-11T15:44:19.861523+0000","last_unstale":"2023-08-11T15:44:55.190486+0000","last_undegraded":"2023-08-11T15:44:55.190486+0000","last_fullsized":"2023-08-11T15:44:55.190486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:22:05.760149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.194869+0000","last_change":"2023-08-11T15:44:19.849283+0000","last_active":"2023-08-11T15:44:55.194869+0000","last_peered":"2023-08-11T15:44:55.194869+0000","last_clean":"2023-08-11T15:44:55.194869+0000","last_became_active":"2023-08-11T15:44:19.848304+0000","last_became_peered":"2023-08-11T15:44:19.848304+0000","last_unstale":"2023-08-11T15:44:55.194869+0000","last_undegraded":"2023-08-11T15:44:55.194869+0000","last_fullsized":"2023-08-11T15:44:55.194869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:03.496778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.190770+0000","last_change":"2023-08-11T15:44:19.853532+0000","last_active":"2023-08-11T15:44:55.190770+0000","last_peered":"2023-08-11T15:44:55.190770+0000","last_clean":"2023-08-11T15:44:55.190770+0000","last_became_active":"2023-08-11T15:44:19.853182+0000","last_became_peered":"2023-08-11T15:44:19.853182+0000","last_unstale":"2023-08-11T15:44:55.190770+0000","last_undegraded":"2023-08-11T15:44:55.190770+0000","last_fullsized":"2023-08-11T15:44:55.190770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T02:39:40.398308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.195608+0000","last_change":"2023-08-11T15:44:19.860571+0000","last_active":"2023-08-11T15:44:55.195608+0000","last_peered":"2023-08-11T15:44:55.195608+0000","last_clean":"2023-08-11T15:44:55.195608+0000","last_became_active":"2023-08-11T15:44:19.858570+0000","last_became_peered":"2023-08-11T15:44:19.858570+0000","last_unstale":"2023-08-11T15:44:55.195608+0000","last_undegraded":"2023-08-11T15:44:55.195608+0000","last_fullsized":"2023-08-11T15:44:55.195608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T03:26:39.509633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-11T15:44:55.192077+0000","last_change":"2023-08-11T15:44:19.865988+0000","last_active":"2023-08-11T15:44:55.192077+0000","last_peered":"2023-08-11T15:44:55.192077+0000","last_clean":"2023-08-11T15:44:55.192077+0000","last_became_active":"2023-08-11T15:44:19.865613+0000","last_became_peered":"2023-08-11T15:44:19.865613+0000","last_unstale":"2023-08-11T15:44:55.192077+0000","last_undegraded":"2023-08-11T15:44:55.192077+0000","last_fullsized":"2023-08-11T15:44:55.192077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T15:44:19.122192+0000","last_clean_scrub_stamp":"2023-08-11T15:44:19.122192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:46:16.828564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62268,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":121,"num_write":73,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":186804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":490303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.626,"15min":0.627},"min":{"1min":0.40799999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.496}]},{"osd":1,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83899999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72799999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.78100000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.442},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.29099999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69899999999999995,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.54300000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85699999999999998,"5min":1.119,"15min":1.119},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91300000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.41199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.88300000000000001,"5min":1.367,"15min":1.367},"last":0.378},{"interface":"front","average":{"1min":0.626,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90300000000000002,"5min":1.631,"15min":1.631},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.4929999999999999,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79100000000000004,"15min":0.80200000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.998,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95799999999999996,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Aug 11 15:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73499999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.90000000000000002,"15min":0.91400000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0229999999999999,"5min":1.742,"15min":1.742},"last":0.752},{"interface":"front","average":{"1min":1.1930000000000001,"5min":0.94599999999999995,"15min":0.90500000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":6.7590000000000003,"5min":6.7590000000000003,"15min":6.7590000000000003},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Aug 11 15:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79700000000000004,"15min":0.79800000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":1.272,"15min":1.272},"last":0.621},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82699999999999996,"15min":0.82999999999999996},"min":{"1min":0.55000000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97199999999999998,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.71399999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1137908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.621},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73699999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.753},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":1.252,"15min":1.252},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.47099999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88200000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.745,"15min":0.755},"min":{"1min":0.52500000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84399999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95599999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79300000000000004}]},{"osd":7,"last update":"Fri Aug 11 15:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77300000000000002,"15min":0.77500000000000002},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.92500000000000004}]},{"osd":9,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1140000000000001,"5min":1.226,"15min":1.226},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0840000000000001,"5min":1.484,"15min":1.484},"last":0.96599999999999997}]},{"osd":11,"last update":"Fri Aug 11 15:45:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.746},"min":{"1min":0.68400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.78900000000000003,"15min":0.78000000000000003},"min":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97999999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":494981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":3.2280000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.1919999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":3.3069999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.095,"5min":1.242,"15min":1.242},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.80600000000000005,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0720000000000001,"5min":1.226,"15min":1.226},"last":3.177}]},{"osd":3,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0680000000000001,"5min":1.171,"15min":1.171},"last":0.96299999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.24,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":3.3700000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.86799999999999999,"15min":0.88700000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0449999999999999,"5min":1.504,"15min":1.504},"last":1.022}]},{"osd":5,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.262,"5min":1.371,"15min":1.371},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.875,"15min":0.88900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1859999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.749}]},{"osd":6,"last update":"Fri Aug 11 15:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.046},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1459999999999999,"5min":1.579,"15min":1.579},"last":0.91100000000000003}]},{"osd":8,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.95799999999999996,"15min":0.97199999999999998},"min":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1679999999999999,"5min":1.323,"15min":1.323},"last":3.27},{"interface":"front","average":{"1min":0.88,"5min":0.995,"15min":1.014},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3149999999999999,"5min":1.429,"15min":1.429},"last":1.169}]},{"osd":10,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.95299999999999996,"15min":0.97099999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2450000000000001,"5min":1.599,"15min":1.599},"last":3.339},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.98899999999999999,"15min":1.0029999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.081,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.302}]},{"osd":11,"last update":"Fri Aug 11 15:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.3260000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.878}]}]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":490576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.60899999999999999,"15min":0.55300000000000005},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.629},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":6.9880000000000004,"5min":6.9880000000000004,"15min":6.9880000000000004},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.627,"15min":0.61199999999999999},"min":{"1min":0.49099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.46100000000000002,"15min":0.437},"min":{"1min":0.39300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.48899999999999999,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.68200000000000005,"15min":0.63500000000000001},"min":{"1min":0.57599999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"last":0.70799999999999996}]},{"osd":4,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65100000000000002,"15min":0.63800000000000001},"min":{"1min":0.54900000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0980000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.90300000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.87}]},{"osd":6,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.72299999999999998,"15min":0.70999999999999996},"min":{"1min":0.55200000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":0.81100000000000005,"15min":0.745},"min":{"1min":0.64200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":7.1079999999999997,"5min":7.1079999999999997,"15min":7.1079999999999997},"last":0.84799999999999998}]},{"osd":7,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.71899999999999997,"15min":0.69699999999999995},"min":{"1min":0.57599999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.70999999999999996},"min":{"1min":0.59599999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.97799999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.90000000000000002,"15min":0.91500000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.083,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":1.018}]},{"osd":10,"last update":"Fri Aug 11 15:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.74299999999999999,"15min":0.72399999999999998},"min":{"1min":0.60099999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.0720000000000001}]},{"osd":11,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0329999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1085349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.45300000000000001,"15min":0.441},"min":{"1min":0.311,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71099999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.44900000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42899999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.436,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77800000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51200000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76100000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46200000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84199999999999997,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88800000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58199999999999996}]},{"osd":6,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.50900000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.66000000000000003}]},{"osd":8,"last update":"Fri Aug 11 15:45:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97899999999999998,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.98699999999999999,"5min":4.8890000000000002,"15min":4.8890000000000002},"last":0.82599999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.59299999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.77200000000000002}]},{"osd":10,"last update":"Fri Aug 11 15:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73999999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.79400000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":486207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":2.077,"15min":2.077},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.008,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0620000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.581,"15min":1.581},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92500000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.67200000000000004},"min":{"1min":0.42899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.752}]},{"osd":4,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.7,"15min":1.7},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Aug 11 15:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0269999999999999,"5min":1.22,"15min":1.22},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1319999999999999,"5min":1.623,"15min":1.623},"last":1.111}]},{"osd":7,"last update":"Fri Aug 11 15:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0740000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.099},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.82599999999999996,"15min":0.79900000000000004},"min":{"1min":0.67000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.099,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0820000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.879,"15min":0.872},"min":{"1min":0.69999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1220000000000001,"5min":1.252,"15min":1.252},"last":1.127},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.76300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1559999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.1539999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96199999999999997,"15min":0.97399999999999998},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.121,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.98999999999999999,"15min":1},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1559999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.1819999999999999}]},{"osd":10,"last update":"Fri Aug 11 15:45:12 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.80400000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.1839999999999999,"5min":1.274,"15min":1.274},"last":1.165},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.97099999999999997,"15min":0.96799999999999997},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.2150000000000001,"5min":1.768,"15min":1.768},"last":1.1739999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:45:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.028}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":547677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.753},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91700000000000004,"5min":4.9020000000000001,"15min":4.9020000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.318,"15min":0.318},"max":{"1min":0.86499999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59599999999999997,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94399999999999995,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.314,"15min":0.314},"max":{"1min":0.78500000000000003,"5min":1.593,"15min":1.593},"last":0.85899999999999999}]},{"osd":3,"last update":"Fri Aug 11 15:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.753},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.024,"15min":1.024},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Aug 11 15:45:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44,"15min":0.41799999999999998},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67800000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.435},"min":{"1min":0.371,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.627,"15min":0.58899999999999997},"min":{"1min":0.58099999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.621,"15min":0.58299999999999996},"min":{"1min":0.59099999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.51100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":1.05,"15min":1.05},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Aug 11 15:45:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51100000000000001,"15min":0.48699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63400000000000001,"15min":0.60199999999999998},"min":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.88100000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.70899999999999996,"15min":0.67700000000000005},"min":{"1min":0.61399999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.56899999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Aug 11 15:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.54800000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1180000000000001,"5min":1.298,"15min":1.298},"last":0.95599999999999996}]},{"osd":10,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64500000000000002,"15min":0.63800000000000001},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80400000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.94099999999999995}]},{"osd":11,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97399999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":486297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 15:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70099999999999996,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.874,"5min":6.0609999999999999,"15min":6.0609999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64200000000000002,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88400000000000001,"5min":6.101,"15min":6.101},"last":0.754}]},{"osd":2,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.56699999999999995},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93400000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.83499999999999996,"15min":0.747},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.7640000000000002,"5min":6.7640000000000002,"15min":6.7640000000000002},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.622},"min":{"1min":0.46300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":1.26,"15min":1.26},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65900000000000003,"15min":0.60499999999999998},"min":{"1min":0.496,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.056,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.872}]},{"osd":5,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.71299999999999997,"15min":0.66100000000000003},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.75,"15min":0.70799999999999996},"min":{"1min":0.64900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.032,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:45:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.61799999999999999}]},{"osd":8,"last update":"Fri Aug 11 15:45:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":0.96099999999999997,"15min":0.91200000000000003},"min":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":6.798,"5min":6.798,"15min":6.798},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.002,"5min":0.89000000000000001,"15min":0.88100000000000001},"min":{"1min":0.78100000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Aug 11 15:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97799999999999998,"5min":1.339,"15min":1.339},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.98499999999999999,"15min":1.0129999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.194,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77800000000000002}]},{"osd":10,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.97299999999999998,"15min":0.93899999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":6.8159999999999998,"5min":6.8159999999999998,"15min":6.8159999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.99299999999999999,"15min":1.008},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.234,"5min":1.472,"15min":1.472},"last":0.84299999999999997}]},{"osd":11,"last update":"Fri Aug 11 15:45:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.024,"15min":0.95399999999999996},"min":{"1min":0.73199999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.91400000000000003,"15min":0.88200000000000001},"min":{"1min":0.83399999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.88800000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":547711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57499999999999996,"15min":0.54600000000000004},"min":{"1min":0.48799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Aug 11 15:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.48599999999999999,"15min":0.45000000000000001},"min":{"1min":0.379,"5min":0.251,"15min":0.251},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.501,"15min":0.47299999999999998},"min":{"1min":0.33000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496}]},{"osd":6,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.75700000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.621},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.625,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.84099999999999997}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Aug 11 15:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.63,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.67700000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.86799999999999999}]},{"osd":11,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.91400000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":486311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.37},{"interface":"front","average":{"1min":0.625,"5min":0.47699999999999998,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68100000000000005,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.82599999999999996}]},{"osd":4,"last update":"Fri Aug 11 15:45:07 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.752,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":5.8659999999999997,"5min":5.8659999999999997,"15min":5.8659999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.66400000000000003},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76500000000000001,"15min":0.747},"min":{"1min":0.64600000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.754},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0700000000000001,"5min":1.179,"15min":1.179},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Aug 11 15:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.72399999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78100000000000003,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.5,"15min":0.5},"max":{"1min":3.9060000000000001,"5min":3.9060000000000001,"15min":3.9060000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.77400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.036,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Aug 11 15:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.81699999999999995}]},{"osd":9,"last update":"Fri Aug 11 15:44:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95099999999999996,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.266,"5min":5.266,"15min":5.266},"last":0.626},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Aug 11 15:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89300000000000002,"15min":0.90400000000000003},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.024,"5min":0.99299999999999999,"15min":0.98799999999999999},"min":{"1min":0.73099999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.77200000000000002}]},{"osd":11,"last update":"Fri Aug 11 15:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.78500000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":490325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 15:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.379},"min":{"1min":0.28299999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.53600000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.38200000000000001},"min":{"1min":0.29399999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56200000000000006,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":1.516,"15min":1.516},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82099999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Aug 11 15:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.47399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84399999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81200000000000006,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.874,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89100000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Aug 11 15:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83599999999999997,"5min":1.552,"15min":1.552},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":1.502,"15min":1.502},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Aug 11 15:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.41899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.84399999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78900000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Aug 11 15:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89500000000000002,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.749,"15min":0.73999999999999999},"min":{"1min":0.58899999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92800000000000005,"5min":1.526,"15min":1.526},"last":0.73799999999999999}]},{"osd":9,"last update":"Fri Aug 11 15:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77100000000000002,"15min":0.76800000000000002},"min":{"1min":0.59699999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.94099999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.54600000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92100000000000004,"5min":1.024,"15min":1.024},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95499999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.754}]},{"osd":11,"last update":"Fri Aug 11 15:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.82499999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1080522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61599999999999999,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":5.359,"15min":5.359},"last":0.73399999999999999}]},{"osd":1,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.92500000000000004,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.91000000000000003,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.377},"min":{"1min":0.35799999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.44,"15min":0.41099999999999998},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.55300000000000005},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61199999999999999,"15min":0.57999999999999996},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Aug 11 15:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60499999999999998,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Aug 11 15:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.032,"15min":1.032},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Aug 11 15:45:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.875},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.65400000000000003,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.88700000000000001}]},{"osd":8,"last update":"Fri Aug 11 15:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60899999999999999,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86799999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67600000000000005,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.376,"15min":0.376},"max":{"1min":0.93799999999999994,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.83399999999999996}]},{"osd":9,"last update":"Fri Aug 11 15:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.91100000000000003}]},{"osd":10,"last update":"Fri Aug 11 15:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.752,"15min":0.73399999999999999},"min":{"1min":0.72199999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.75,"15min":0.73199999999999998},"min":{"1min":0.68700000000000006,"5min":0.505,"15min":0.505},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T15:45:48.220 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-08-11T15:45:48.221 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-11T15:45:48.222 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-08-11T15:45:48.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph health --format=json 2023-08-11T15:45:48.955 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:49.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:49 smithi134 ceph-mon[130447]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:49.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:49 smithi134 ceph-mon[130447]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:49.540 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:49 smithi032 ceph-mon[97103]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:49.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:49 smithi032 ceph-mon[97103]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:49.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:49 smithi192 ceph-mon[130806]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T15:45:49.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:49 smithi192 ceph-mon[130806]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 -- 172.21.15.32:0/524046171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190104730 msgr2=0x7fb190104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 --2- 172.21.15.32:0/524046171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190104730 0x7fb190104b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009990 tx=0x7fb18c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 -- 172.21.15.32:0/524046171 shutdown_connections 2023-08-11T15:45:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 --2- 172.21.15.32:0/524046171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190104730 0x7fb190104b10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 --2- 172.21.15.32:0/524046171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb190100220 0x7fb19010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 --2- 172.21.15.32:0/524046171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1900ff860 0x7fb1900ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 -- 172.21.15.32:0/524046171 >> 172.21.15.32:0/524046171 conn(0x7fb190076920 msgr2=0x7fb190076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.441+0000 7fb197c7b700 1 -- 172.21.15.32:0/524046171 shutdown_connections 2023-08-11T15:45:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.442+0000 7fb197c7b700 1 -- 172.21.15.32:0/524046171 wait complete. 2023-08-11T15:45:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.442+0000 7fb197c7b700 1 Processor -- start 2023-08-11T15:45:50.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb197c7b700 1 -- start start 2023-08-11T15:45:50.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb197c7b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1900ff860 0x7fb19019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:50.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb197c7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190100220 0x7fb19019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:50.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb197c7b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190104730 0x7fb190198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:50.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb197c7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190199140 con 0x7fb190100220 2023-08-11T15:45:50.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb197c7b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901992b0 con 0x7fb1900ff860 2023-08-11T15:45:50.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb197c7b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190199470 con 0x7fb190104730 2023-08-11T15:45:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.443+0000 7fb195216700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190100220 0x7fb19019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195a17700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1900ff860 0x7fb19019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195216700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190100220 0x7fb19019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38664/0 (socket says 172.21.15.32:38664) 2023-08-11T15:45:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195216700 1 -- 172.21.15.32:0/274712891 learned_addr learned my addr 172.21.15.32:0/274712891 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb196218700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190104730 0x7fb190198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195a17700 1 -- 172.21.15.32:0/274712891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190104730 msgr2=0x7fb190198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195a17700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190104730 0x7fb190198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195a17700 1 -- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190100220 msgr2=0x7fb19019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195a17700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190100220 0x7fb19019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195a17700 1 -- 172.21.15.32:0/274712891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0095f0 con 0x7fb1900ff860 2023-08-11T15:45:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195216700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190100220 0x7fb19019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.444+0000 7fb195a17700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1900ff860 0x7fb19019ea50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009990 tx=0x7fb18c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:50.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.445+0000 7fb186ffd700 1 -- 172.21.15.32:0/274712891 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c01d070 con 0x7fb1900ff860 2023-08-11T15:45:50.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.445+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1901996f0 con 0x7fb1900ff860 2023-08-11T15:45:50.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.445+0000 7fb186ffd700 1 -- 172.21.15.32:0/274712891 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c021410 con 0x7fb1900ff860 2023-08-11T15:45:50.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.445+0000 7fb186ffd700 1 -- 172.21.15.32:0/274712891 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c017650 con 0x7fb1900ff860 2023-08-11T15:45:50.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.445+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1901a5a40 con 0x7fb1900ff860 2023-08-11T15:45:50.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.446+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19004e620 con 0x7fb1900ff860 2023-08-11T15:45:50.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.448+0000 7fb186ffd700 1 -- 172.21.15.32:0/274712891 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb18c021810 con 0x7fb1900ff860 2023-08-11T15:45:50.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.449+0000 7fb186ffd700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb17c09ffe0 0x7fb17c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:50.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.449+0000 7fb186ffd700 1 -- 172.21.15.32:0/274712891 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb18c0ccbc0 con 0x7fb1900ff860 2023-08-11T15:45:50.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.449+0000 7fb195216700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb17c09ffe0 0x7fb17c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:50.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.450+0000 7fb195216700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb17c09ffe0 0x7fb17c0a24a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb190105e40 tx=0x7fb180009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:50.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.451+0000 7fb186ffd700 1 -- 172.21.15.32:0/274712891 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb18c096510 con 0x7fb1900ff860 2023-08-11T15:45:50.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.702+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb190199fe0 con 0x7fb1900ff860 2023-08-11T15:45:50.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.702+0000 7fb186ffd700 1 -- 172.21.15.32:0/274712891 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb18c099b30 con 0x7fb1900ff860 2023-08-11T15:45:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:50.706 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-11T15:45:50.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb17c09ffe0 msgr2=0x7fb17c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb17c09ffe0 0x7fb17c0a24a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb190105e40 tx=0x7fb180009260 comp rx=0 tx=0).stop 2023-08-11T15:45:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1900ff860 msgr2=0x7fb19019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1900ff860 0x7fb19019ea50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009990 tx=0x7fb18c009580 comp rx=0 tx=0).stop 2023-08-11T15:45:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 shutdown_connections 2023-08-11T15:45:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb17c09ffe0 0x7fb17c0a24a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190104730 0x7fb190198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1900ff860 0x7fb19019ea50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 --2- 172.21.15.32:0/274712891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb190100220 0x7fb19019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:50.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 >> 172.21.15.32:0/274712891 conn(0x7fb190076920 msgr2=0x7fb1900fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:50.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 shutdown_connections 2023-08-11T15:45:50.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:50.705+0000 7fb197c7b700 1 -- 172.21.15.32:0/274712891 wait complete. 2023-08-11T15:45:51.152 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-08-11T15:45:51.153 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:45:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.362+0000 7f26589ff700 1 -- 172.21.15.32:0/445989362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654100920 msgr2=0x7f2654100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.362+0000 7f26589ff700 1 --2- 172.21.15.32:0/445989362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654100920 0x7f2654100d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26440099c0 tx=0x7f2644009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.363+0000 7f26589ff700 1 -- 172.21.15.32:0/445989362 shutdown_connections 2023-08-11T15:45:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.363+0000 7f26589ff700 1 --2- 172.21.15.32:0/445989362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654100920 0x7f2654100d40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.363+0000 7f26589ff700 1 --2- 172.21.15.32:0/445989362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26541024b0 0x7f2654108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.363+0000 7f26589ff700 1 --2- 172.21.15.32:0/445989362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2654101b10 0x7f2654101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.363+0000 7f26589ff700 1 -- 172.21.15.32:0/445989362 >> 172.21.15.32:0/445989362 conn(0x7f26540fbe50 msgr2=0x7f26540fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.363+0000 7f26589ff700 1 -- 172.21.15.32:0/445989362 shutdown_connections 2023-08-11T15:45:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.363+0000 7f26589ff700 1 -- 172.21.15.32:0/445989362 wait complete. 2023-08-11T15:45:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 Processor -- start 2023-08-11T15:45:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 -- start start 2023-08-11T15:45:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2654100920 0x7f2654198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654101b10 0x7f2654198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26541024b0 0x7f265419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26541a0390 con 0x7f2654101b10 2023-08-11T15:45:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26541a04d0 con 0x7f2654100920 2023-08-11T15:45:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.364+0000 7f26589ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26541a0660 con 0x7f26541024b0 2023-08-11T15:45:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654101b10 0x7f2654198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654101b10 0x7f2654198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38690/0 (socket says 172.21.15.32:38690) 2023-08-11T15:45:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 -- 172.21.15.32:0/2224654281 learned_addr learned my addr 172.21.15.32:0/2224654281 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f265259c700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2654100920 0x7f2654198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f2652d9d700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26541024b0 0x7f265419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 -- 172.21.15.32:0/2224654281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26541024b0 msgr2=0x7f265419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26541024b0 0x7f265419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 -- 172.21.15.32:0/2224654281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2654100920 msgr2=0x7f2654198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2654100920 0x7f2654198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 -- 172.21.15.32:0/2224654281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440095f0 con 0x7f2654101b10 2023-08-11T15:45:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.365+0000 7f264bfff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654101b10 0x7f2654198d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d980 tx=0x7f263c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.366+0000 7f264b7fe700 1 -- 172.21.15.32:0/2224654281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c004220 con 0x7f2654101b10 2023-08-11T15:45:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.366+0000 7f264b7fe700 1 -- 172.21.15.32:0/2224654281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f263c007640 con 0x7f2654101b10 2023-08-11T15:45:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.366+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26541a0940 con 0x7f2654101b10 2023-08-11T15:45:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.366+0000 7f264b7fe700 1 -- 172.21.15.32:0/2224654281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c005690 con 0x7f2654101b10 2023-08-11T15:45:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.366+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26541a0ea0 con 0x7f2654101b10 2023-08-11T15:45:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.367+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2654192f70 con 0x7f2654101b10 2023-08-11T15:45:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.369+0000 7f264b7fe700 1 -- 172.21.15.32:0/2224654281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f263c004750 con 0x7f2654101b10 2023-08-11T15:45:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.370+0000 7f264b7fe700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26400a00a0 0x7f26400a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.370+0000 7f264b7fe700 1 -- 172.21.15.32:0/2224654281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f263c0cb9b0 con 0x7f2654101b10 2023-08-11T15:45:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.371+0000 7f265259c700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26400a00a0 0x7f26400a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.371+0000 7f265259c700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26400a00a0 0x7f26400a2560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2644009990 tx=0x7f2644005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.373+0000 7f264b7fe700 1 -- 172.21.15.32:0/2224654281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f263c095270 con 0x7f2654101b10 2023-08-11T15:45:51.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:51 smithi134 ceph-mon[130447]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:51.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:51 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/274712891' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-11T15:45:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:51 smithi032 ceph-mon[97103]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:51 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/274712891' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-11T15:45:51.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:51 smithi192 ceph-mon[130806]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:51.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:51 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/274712891' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-11T15:45:51.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.585+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f265410c180 con 0x7f2654101b10 2023-08-11T15:45:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.587+0000 7f264b7fe700 1 -- 172.21.15.32:0/2224654281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f263c016020 con 0x7f2654101b10 2023-08-11T15:45:51.590 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:51.590 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:55.185701+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:45:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26400a00a0 msgr2=0x7f26400a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26400a00a0 0x7f26400a2560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2644009990 tx=0x7f2644005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654101b10 msgr2=0x7f2654198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654101b10 0x7f2654198d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d980 tx=0x7f263c00dc90 comp rx=0 tx=0).stop 2023-08-11T15:45:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 shutdown_connections 2023-08-11T15:45:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f26400a00a0 0x7f26400a2560 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26541024b0 0x7f265419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2654100920 0x7f2654198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 --2- 172.21.15.32:0/2224654281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2654101b10 0x7f2654198d50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.589+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 >> 172.21.15.32:0/2224654281 conn(0x7f26540fbe50 msgr2=0x7f2654107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:51.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.590+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 shutdown_connections 2023-08-11T15:45:51.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.590+0000 7f26589ff700 1 -- 172.21.15.32:0/2224654281 wait complete. 2023-08-11T15:45:51.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:45:51.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:45:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 -- 172.21.15.32:0/3337571786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0100930 msgr2=0x7fe1e0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/3337571786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0100930 0x7fe1e0100d50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0099c0 tx=0x7fe1dc009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 -- 172.21.15.32:0/3337571786 shutdown_connections 2023-08-11T15:45:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/3337571786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0100930 0x7fe1e0100d50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/3337571786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e01024c0 0x7fe1e0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/3337571786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0101b20 0x7fe1e0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 -- 172.21.15.32:0/3337571786 >> 172.21.15.32:0/3337571786 conn(0x7fe1e00fbea0 msgr2=0x7fe1e00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:51.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 -- 172.21.15.32:0/3337571786 shutdown_connections 2023-08-11T15:45:51.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.818+0000 7fe1e83ff700 1 -- 172.21.15.32:0/3337571786 wait complete. 2023-08-11T15:45:51.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 Processor -- start 2023-08-11T15:45:51.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 -- start start 2023-08-11T15:45:51.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0100930 0x7fe1e00746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0101b20 0x7fe1e0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e01024c0 0x7fe1e0073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00739f0 con 0x7fe1e01024c0 2023-08-11T15:45:51.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e0073b60 con 0x7fe1e0100930 2023-08-11T15:45:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.819+0000 7fe1e83ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01a0600 con 0x7fe1e0101b20 2023-08-11T15:45:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e01024c0 0x7fe1e0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e01024c0 0x7fe1e0073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38716/0 (socket says 172.21.15.32:38716) 2023-08-11T15:45:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 -- 172.21.15.32:0/950833866 learned_addr learned my addr 172.21.15.32:0/950833866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e619b700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0100930 0x7fe1e00746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e599a700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0101b20 0x7fe1e0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 -- 172.21.15.32:0/950833866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0101b20 msgr2=0x7fe1e0072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0101b20 0x7fe1e0072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 -- 172.21.15.32:0/950833866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0100930 msgr2=0x7fe1e00746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0100930 0x7fe1e00746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 -- 172.21.15.32:0/950833866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0095f0 con 0x7fe1e01024c0 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e599a700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0101b20 0x7fe1e0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.820+0000 7fe1e699c700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e01024c0 0x7fe1e0073250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800ef30 tx=0x7fe1d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.821+0000 7fe1d77fe700 1 -- 172.21.15.32:0/950833866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8018070 con 0x7fe1e01024c0 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.821+0000 7fe1d77fe700 1 -- 172.21.15.32:0/950833866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1d800f040 con 0x7fe1e01024c0 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.821+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e01a0880 con 0x7fe1e01024c0 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.821+0000 7fe1d77fe700 1 -- 172.21.15.32:0/950833866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800cdb0 con 0x7fe1e01024c0 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.821+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e01a0d40 con 0x7fe1e01024c0 2023-08-11T15:45:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.822+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e0107810 con 0x7fe1e01024c0 2023-08-11T15:45:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.824+0000 7fe1d77fe700 1 -- 172.21.15.32:0/950833866 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fe1d8003a00 con 0x7fe1e01024c0 2023-08-11T15:45:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.826+0000 7fe1d77fe700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1cc09ffe0 0x7fe1cc0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.826+0000 7fe1d77fe700 1 -- 172.21.15.32:0/950833866 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe1d80cb380 con 0x7fe1e01024c0 2023-08-11T15:45:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.826+0000 7fe1e619b700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1cc09ffe0 0x7fe1cc0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.827+0000 7fe1e619b700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1cc09ffe0 0x7fe1cc0a24a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009990 tx=0x7fe1dc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:51.830+0000 7fe1d77fe700 1 -- 172.21.15.32:0/950833866 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe1d8094cd0 con 0x7fe1e01024c0 2023-08-11T15:45:52.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.040+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1e010c3e0 con 0x7fe1e01024c0 2023-08-11T15:45:52.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.040+0000 7fe1d77fe700 1 -- 172.21.15.32:0/950833866 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1d80982f0 con 0x7fe1e01024c0 2023-08-11T15:45:52.043 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:45:52.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.042+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1cc09ffe0 msgr2=0x7fe1cc0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.042+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1cc09ffe0 0x7fe1cc0a24a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009990 tx=0x7fe1dc005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.042+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e01024c0 msgr2=0x7fe1e0073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.042+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e01024c0 0x7fe1e0073250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800ef30 tx=0x7fe1d800c3a0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 shutdown_connections 2023-08-11T15:45:52.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe1cc09ffe0 0x7fe1cc0a24a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e01024c0 0x7fe1e0073250 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0100930 0x7fe1e00746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 --2- 172.21.15.32:0/950833866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0101b20 0x7fe1e0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 >> 172.21.15.32:0/950833866 conn(0x7fe1e00fbea0 msgr2=0x7fe1e00fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:52.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 shutdown_connections 2023-08-11T15:45:52.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.043+0000 7fe1e83ff700 1 -- 172.21.15.32:0/950833866 wait complete. 2023-08-11T15:45:52.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:45:52.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 -- 172.21.15.32:0/63852205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534100930 msgr2=0x7fa534100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 --2- 172.21.15.32:0/63852205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534100930 0x7fa534100d50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa524009920 tx=0x7fa524009c30 comp rx=0 tx=0).stop 2023-08-11T15:45:52.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 -- 172.21.15.32:0/63852205 shutdown_connections 2023-08-11T15:45:52.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 --2- 172.21.15.32:0/63852205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5341024c0 0x7fa534108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 --2- 172.21.15.32:0/63852205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534100930 0x7fa534100d50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 --2- 172.21.15.32:0/63852205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 0x7fa534101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 -- 172.21.15.32:0/63852205 >> 172.21.15.32:0/63852205 conn(0x7fa5340fbea0 msgr2=0x7fa5340fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:52.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 -- 172.21.15.32:0/63852205 shutdown_connections 2023-08-11T15:45:52.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.273+0000 7fa5393ff700 1 -- 172.21.15.32:0/63852205 wait complete. 2023-08-11T15:45:52.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.274+0000 7fa5393ff700 1 Processor -- start 2023-08-11T15:45:52.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.274+0000 7fa5393ff700 1 -- start start 2023-08-11T15:45:52.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5393ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa534100930 0x7fa534196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5393ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 0x7fa534196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5393ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5341024c0 0x7fa53419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5393ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53419e200 con 0x7fa534100930 2023-08-11T15:45:52.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5393ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53419e340 con 0x7fa5341024c0 2023-08-11T15:45:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5393ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53419e4d0 con 0x7fa534101b20 2023-08-11T15:45:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 0x7fa534196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 0x7fa534196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:60664/0 (socket says 172.21.15.32:60664) 2023-08-11T15:45:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 -- 172.21.15.32:0/855249066 learned_addr learned my addr 172.21.15.32:0/855249066 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa532ffd700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa534100930 0x7fa534196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5337fe700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5341024c0 0x7fa53419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 -- 172.21.15.32:0/855249066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5341024c0 msgr2=0x7fa53419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5341024c0 0x7fa53419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 -- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa534100930 msgr2=0x7fa534196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa534100930 0x7fa534196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.275+0000 7fa5327fc700 1 -- 172.21.15.32:0/855249066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa524009520 con 0x7fa534101b20 2023-08-11T15:45:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.276+0000 7fa5327fc700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 0x7fa534196bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9c0 tx=0x7fa52800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.276+0000 7fa51bfff700 1 -- 172.21.15.32:0/855249066 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528004a30 con 0x7fa534101b20 2023-08-11T15:45:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.276+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53419e7b0 con 0x7fa534101b20 2023-08-11T15:45:52.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.276+0000 7fa51bfff700 1 -- 172.21.15.32:0/855249066 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa52800cc80 con 0x7fa534101b20 2023-08-11T15:45:52.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.276+0000 7fa51bfff700 1 -- 172.21.15.32:0/855249066 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5280054e0 con 0x7fa534101b20 2023-08-11T15:45:52.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.276+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53419ecc0 con 0x7fa534101b20 2023-08-11T15:45:52.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.277+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53410c140 con 0x7fa534101b20 2023-08-11T15:45:52.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.279+0000 7fa51bfff700 1 -- 172.21.15.32:0/855249066 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fa528004b90 con 0x7fa534101b20 2023-08-11T15:45:52.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.280+0000 7fa51bfff700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa51c09ffe0 0x7fa51c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.281+0000 7fa51bfff700 1 -- 172.21.15.32:0/855249066 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa528014070 con 0x7fa534101b20 2023-08-11T15:45:52.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.281+0000 7fa532ffd700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa51c09ffe0 0x7fa51c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.281+0000 7fa532ffd700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa51c09ffe0 0x7fa51c0a24a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5240098f0 tx=0x7fa524019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:52.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.282+0000 7fa51bfff700 1 -- 172.21.15.32:0/855249066 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa528095280 con 0x7fa534101b20 2023-08-11T15:45:52.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:52 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2224654281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:52.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:52 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/950833866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:45:52.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.494+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5341078c0 con 0x7fa534101b20 2023-08-11T15:45:52.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.495+0000 7fa51bfff700 1 -- 172.21.15.32:0/855249066 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5280988a0 con 0x7fa534101b20 2023-08-11T15:45:52.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:45:52.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa51c09ffe0 msgr2=0x7fa51c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa51c09ffe0 0x7fa51c0a24a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5240098f0 tx=0x7fa524019040 comp rx=0 tx=0).stop 2023-08-11T15:45:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 msgr2=0x7fa534196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 0x7fa534196bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9c0 tx=0x7fa52800ecd0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 shutdown_connections 2023-08-11T15:45:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fa51c09ffe0 0x7fa51c0a24a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa534100930 0x7fa534196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5341024c0 0x7fa53419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 --2- 172.21.15.32:0/855249066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa534101b20 0x7fa534196bc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 >> 172.21.15.32:0/855249066 conn(0x7fa5340fbea0 msgr2=0x7fa5340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 shutdown_connections 2023-08-11T15:45:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.497+0000 7fa5393ff700 1 -- 172.21.15.32:0/855249066 wait complete. 2023-08-11T15:45:52.516 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:52 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2224654281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:52.517 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:52 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/950833866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:45:52.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:45:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:52 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2224654281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:52 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/950833866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:45:52.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 -- 172.21.15.32:0/3939339237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c100910 msgr2=0x7fb69c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/3939339237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c100910 0x7fb69c100d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0099c0 tx=0x7fb68c009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 -- 172.21.15.32:0/3939339237 shutdown_connections 2023-08-11T15:45:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/3939339237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c1024a0 0x7fb69c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/3939339237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c100910 0x7fb69c100d30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/3939339237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb69c101b00 0x7fb69c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 -- 172.21.15.32:0/3939339237 >> 172.21.15.32:0/3939339237 conn(0x7fb69c0fbe60 msgr2=0x7fb69c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 -- 172.21.15.32:0/3939339237 shutdown_connections 2023-08-11T15:45:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.723+0000 7fb6a09ff700 1 -- 172.21.15.32:0/3939339237 wait complete. 2023-08-11T15:45:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.724+0000 7fb6a09ff700 1 Processor -- start 2023-08-11T15:45:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb6a09ff700 1 -- start start 2023-08-11T15:45:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb6a09ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c100910 0x7fb69c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb6a09ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c101b00 0x7fb69c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb6a09ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb69c1024a0 0x7fb69c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb6a09ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c19a530 con 0x7fb69c100910 2023-08-11T15:45:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb6a09ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c073790 con 0x7fb69c101b00 2023-08-11T15:45:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb6a09ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c073950 con 0x7fb69c1024a0 2023-08-11T15:45:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb69a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c100910 0x7fb69c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb69a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c100910 0x7fb69c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38760/0 (socket says 172.21.15.32:38760) 2023-08-11T15:45:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb69a59c700 1 -- 172.21.15.32:0/2977486241 learned_addr learned my addr 172.21.15.32:0/2977486241 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb693fff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c101b00 0x7fb69c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.725+0000 7fb69ad9d700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb69c1024a0 0x7fb69c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb69a59c700 1 -- 172.21.15.32:0/2977486241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb69c1024a0 msgr2=0x7fb69c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb69a59c700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb69c1024a0 0x7fb69c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb69a59c700 1 -- 172.21.15.32:0/2977486241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c101b00 msgr2=0x7fb69c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb69a59c700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c101b00 0x7fb69c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb69a59c700 1 -- 172.21.15.32:0/2977486241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c0095f0 con 0x7fb69c100910 2023-08-11T15:45:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb69a59c700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c100910 0x7fb69c0746d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009990 tx=0x7fb68c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb6937fe700 1 -- 172.21.15.32:0/2977486241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c01d070 con 0x7fb69c100910 2023-08-11T15:45:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb6937fe700 1 -- 172.21.15.32:0/2977486241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb68c005030 con 0x7fb69c100910 2023-08-11T15:45:52.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c1a4ee0 con 0x7fb69c100910 2023-08-11T15:45:52.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb6937fe700 1 -- 172.21.15.32:0/2977486241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c004a70 con 0x7fb69c100910 2023-08-11T15:45:52.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.726+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c1a5250 con 0x7fb69c100910 2023-08-11T15:45:52.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.727+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c10c170 con 0x7fb69c100910 2023-08-11T15:45:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.729+0000 7fb6937fe700 1 -- 172.21.15.32:0/2977486241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb68c005410 con 0x7fb69c100910 2023-08-11T15:45:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.731+0000 7fb6937fe700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb68809ffe0 0x7fb6880a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.731+0000 7fb6937fe700 1 -- 172.21.15.32:0/2977486241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb68c0cc4a0 con 0x7fb69c100910 2023-08-11T15:45:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.731+0000 7fb693fff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb68809ffe0 0x7fb6880a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:52.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.732+0000 7fb693fff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb68809ffe0 0x7fb6880a24a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb684009c60 tx=0x7fb684009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.735+0000 7fb6937fe700 1 -- 172.21.15.32:0/2977486241 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb68c095df0 con 0x7fb69c100910 2023-08-11T15:45:52.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.948+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb69c1078c0 con 0x7fb69c100910 2023-08-11T15:45:52.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.948+0000 7fb6937fe700 1 -- 172.21.15.32:0/2977486241 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb68c099410 con 0x7fb69c100910 2023-08-11T15:45:52.951 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:45:52.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.950+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb68809ffe0 msgr2=0x7fb6880a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb68809ffe0 0x7fb6880a24a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb684009c60 tx=0x7fb684009260 comp rx=0 tx=0).stop 2023-08-11T15:45:52.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c100910 msgr2=0x7fb69c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:52.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c100910 0x7fb69c0746d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009990 tx=0x7fb68c00f690 comp rx=0 tx=0).stop 2023-08-11T15:45:52.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 shutdown_connections 2023-08-11T15:45:52.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb68809ffe0 0x7fb6880a24a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb69c100910 0x7fb69c0746d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb69c1024a0 0x7fb69c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 --2- 172.21.15.32:0/2977486241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c101b00 0x7fb69c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:52.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 >> 172.21.15.32:0/2977486241 conn(0x7fb69c0fbe60 msgr2=0x7fb69c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:52.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 shutdown_connections 2023-08-11T15:45:52.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:52.951+0000 7fb6a09ff700 1 -- 172.21.15.32:0/2977486241 wait complete. 2023-08-11T15:45:52.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:45:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 -- 172.21.15.32:0/562526110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff218100910 msgr2=0x7ff218100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 --2- 172.21.15.32:0/562526110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff218100910 0x7ff218100d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2140099c0 tx=0x7ff214009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 -- 172.21.15.32:0/562526110 shutdown_connections 2023-08-11T15:45:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 --2- 172.21.15.32:0/562526110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2181024a0 0x7ff218108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 --2- 172.21.15.32:0/562526110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff218100910 0x7ff218100d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 --2- 172.21.15.32:0/562526110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff218101b00 0x7ff218101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 -- 172.21.15.32:0/562526110 >> 172.21.15.32:0/562526110 conn(0x7ff2180fbe60 msgr2=0x7ff2180fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 -- 172.21.15.32:0/562526110 shutdown_connections 2023-08-11T15:45:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.179+0000 7ff21fbff700 1 -- 172.21.15.32:0/562526110 wait complete. 2023-08-11T15:45:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.180+0000 7ff21fbff700 1 Processor -- start 2023-08-11T15:45:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21fbff700 1 -- start start 2023-08-11T15:45:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21fbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff218100910 0x7ff218198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21fbff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff218101b00 0x7ff218198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21fbff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2181024a0 0x7ff21819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21fbff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181a0400 con 0x7ff218101b00 2023-08-11T15:45:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21fbff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181a0540 con 0x7ff2181024a0 2023-08-11T15:45:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21fbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181a06d0 con 0x7ff218100910 2023-08-11T15:45:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21d19a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff218101b00 0x7ff218198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21d99b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff218100910 0x7ff218198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21d19a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff218101b00 0x7ff218198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38788/0 (socket says 172.21.15.32:38788) 2023-08-11T15:45:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21d19a700 1 -- 172.21.15.32:0/3890471773 learned_addr learned my addr 172.21.15.32:0/3890471773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.181+0000 7ff21e19c700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2181024a0 0x7ff21819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21d19a700 1 -- 172.21.15.32:0/3890471773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff218100910 msgr2=0x7ff218198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21d19a700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff218100910 0x7ff218198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21d19a700 1 -- 172.21.15.32:0/3890471773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2181024a0 msgr2=0x7ff21819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21d19a700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2181024a0 0x7ff21819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21d19a700 1 -- 172.21.15.32:0/3890471773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2140095f0 con 0x7ff218101b00 2023-08-11T15:45:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21d99b700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff218100910 0x7ff218198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21d19a700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff218101b00 0x7ff218198dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff20800e9a0 tx=0x7ff20800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff20effd700 1 -- 172.21.15.32:0/3890471773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff208009790 con 0x7ff218101b00 2023-08-11T15:45:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff20effd700 1 -- 172.21.15.32:0/3890471773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff208010430 con 0x7ff218101b00 2023-08-11T15:45:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff20effd700 1 -- 172.21.15.32:0/3890471773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff208018680 con 0x7ff218101b00 2023-08-11T15:45:53.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.182+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2181a09b0 con 0x7ff218101b00 2023-08-11T15:45:53.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.183+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2181a0e70 con 0x7ff218101b00 2023-08-11T15:45:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.184+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21810c190 con 0x7ff218101b00 2023-08-11T15:45:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.185+0000 7ff20effd700 1 -- 172.21.15.32:0/3890471773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff2080105a0 con 0x7ff218101b00 2023-08-11T15:45:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.187+0000 7ff20effd700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff2040a0060 0x7ff2040a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.187+0000 7ff20effd700 1 -- 172.21.15.32:0/3890471773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff208014070 con 0x7ff218101b00 2023-08-11T15:45:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.187+0000 7ff21d99b700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff2040a0060 0x7ff2040a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.188+0000 7ff21d99b700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff2040a0060 0x7ff2040a2520 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff214009990 tx=0x7ff21401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.191+0000 7ff20effd700 1 -- 172.21.15.32:0/3890471773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff208095000 con 0x7ff218101b00 2023-08-11T15:45:53.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:53 smithi032 ceph-mon[97103]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:53.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:53 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/855249066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:45:53.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:53 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2977486241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:45:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.401+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2181a1060 con 0x7ff218101b00 2023-08-11T15:45:53.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.402+0000 7ff20effd700 1 -- 172.21.15.32:0/3890471773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff208098620 con 0x7ff218101b00 2023-08-11T15:45:53.404 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:45:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff2040a0060 msgr2=0x7ff2040a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff2040a0060 0x7ff2040a2520 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff214009990 tx=0x7ff21401a040 comp rx=0 tx=0).stop 2023-08-11T15:45:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff218101b00 msgr2=0x7ff218198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff218101b00 0x7ff218198dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff20800e9a0 tx=0x7ff20800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 shutdown_connections 2023-08-11T15:45:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff2040a0060 0x7ff2040a2520 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2181024a0 0x7ff21819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff218100910 0x7ff218198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 --2- 172.21.15.32:0/3890471773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff218101b00 0x7ff218198dc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 >> 172.21.15.32:0/3890471773 conn(0x7ff2180fbe60 msgr2=0x7ff2180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 shutdown_connections 2023-08-11T15:45:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.404+0000 7ff21fbff700 1 -- 172.21.15.32:0/3890471773 wait complete. 2023-08-11T15:45:53.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:45:53.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:53 smithi134 ceph-mon[130447]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:53 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/855249066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:45:53.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:53 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2977486241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:45:53.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:53 smithi192 ceph-mon[130806]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:45:53.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:53 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/855249066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:45:53.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:53 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2977486241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:45:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1792926241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4101b10 msgr2=0x7f35f4101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1792926241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4101b10 0x7f35f4101f70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35e40099c0 tx=0x7f35e4009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1792926241 shutdown_connections 2023-08-11T15:45:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1792926241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f41024b0 0x7f35f4108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1792926241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f4100920 0x7f35f4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1792926241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4101b10 0x7f35f4101f70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1792926241 >> 172.21.15.32:0/1792926241 conn(0x7f35f40fbe50 msgr2=0x7f35f40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1792926241 shutdown_connections 2023-08-11T15:45:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.627+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1792926241 wait complete. 2023-08-11T15:45:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.628+0000 7f35fb7ff700 1 Processor -- start 2023-08-11T15:45:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.628+0000 7f35fb7ff700 1 -- start start 2023-08-11T15:45:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35fb7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4100920 0x7f35f40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35fb7ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4101b10 0x7f35f4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35fb7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f41024b0 0x7f35f4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35fb7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40739f0 con 0x7f35f4100920 2023-08-11T15:45:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35fb7ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f4073b60 con 0x7f35f4101b10 2023-08-11T15:45:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35fb7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41a0620 con 0x7f35f41024b0 2023-08-11T15:45:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f41024b0 0x7f35f4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f41024b0 0x7f35f4073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:60736/0 (socket says 172.21.15.32:60736) 2023-08-11T15:45:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 -- 172.21.15.32:0/1007390722 learned_addr learned my addr 172.21.15.32:0/1007390722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f8d9a700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4101b10 0x7f35f4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f959b700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4100920 0x7f35f40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 -- 172.21.15.32:0/1007390722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4101b10 msgr2=0x7f35f4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4101b10 0x7f35f4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 -- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4100920 msgr2=0x7f35f40746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4100920 0x7f35f40746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f9d9c700 1 -- 172.21.15.32:0/1007390722 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40095f0 con 0x7f35f41024b0 2023-08-11T15:45:53.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.629+0000 7f35f959b700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4100920 0x7f35f40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:53.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.630+0000 7f35f9d9c700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f41024b0 0x7f35f4073250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00ef30 tx=0x7f35ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:53.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.630+0000 7f35ea7fc700 1 -- 172.21.15.32:0/1007390722 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec018070 con 0x7f35f41024b0 2023-08-11T15:45:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.630+0000 7f35ea7fc700 1 -- 172.21.15.32:0/1007390722 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ec00f040 con 0x7f35f41024b0 2023-08-11T15:45:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.630+0000 7f35ea7fc700 1 -- 172.21.15.32:0/1007390722 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec00cd00 con 0x7f35f41024b0 2023-08-11T15:45:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.630+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f41a08a0 con 0x7f35f41024b0 2023-08-11T15:45:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.630+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f41a0d60 con 0x7f35f41024b0 2023-08-11T15:45:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.631+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f410c130 con 0x7f35f41024b0 2023-08-11T15:45:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.633+0000 7f35ea7fc700 1 -- 172.21.15.32:0/1007390722 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f35ec003a00 con 0x7f35f41024b0 2023-08-11T15:45:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.635+0000 7f35ea7fc700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35e00a00a0 0x7f35e00a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.635+0000 7f35ea7fc700 1 -- 172.21.15.32:0/1007390722 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f35ec0cbbe0 con 0x7f35f41024b0 2023-08-11T15:45:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.635+0000 7f35f959b700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35e00a00a0 0x7f35e00a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.635+0000 7f35f959b700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35e00a00a0 0x7f35e00a2560 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f35f4101970 tx=0x7f35f0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.637+0000 7f35ea7fc700 1 -- 172.21.15.32:0/1007390722 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f35ec095530 con 0x7f35f41024b0 2023-08-11T15:45:53.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.894+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35f4107840 con 0x7f35f41024b0 2023-08-11T15:45:53.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.895+0000 7f35ea7fc700 1 -- 172.21.15.32:0/1007390722 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f35ec098b50 con 0x7f35f41024b0 2023-08-11T15:45:53.898 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:53.898 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:52.172791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:45:53.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35e00a00a0 msgr2=0x7f35e00a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35e00a00a0 0x7f35e00a2560 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f35f4101970 tx=0x7f35f0009260 comp rx=0 tx=0).stop 2023-08-11T15:45:53.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f41024b0 msgr2=0x7f35f4073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:53.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f41024b0 0x7f35f4073250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00ef30 tx=0x7f35ec00c3a0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 shutdown_connections 2023-08-11T15:45:53.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f35e00a00a0 0x7f35e00a2560 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f4100920 0x7f35f40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35f41024b0 0x7f35f4073250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 --2- 172.21.15.32:0/1007390722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4101b10 0x7f35f4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:53.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 >> 172.21.15.32:0/1007390722 conn(0x7f35f40fbe50 msgr2=0x7f35f40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:53.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 shutdown_connections 2023-08-11T15:45:53.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:53.898+0000 7f35fb7ff700 1 -- 172.21.15.32:0/1007390722 wait complete. 2023-08-11T15:45:53.902 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-08-11T15:45:53.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:45:54.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 -- 172.21.15.32:0/1767857792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090101380 msgr2=0x7fd090103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 --2- 172.21.15.32:0/1767857792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090101380 0x7fd090103770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd080009990 tx=0x7fd080009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 -- 172.21.15.32:0/1767857792 shutdown_connections 2023-08-11T15:45:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 --2- 172.21.15.32:0/1767857792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0901065e0 0x7fd090108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 --2- 172.21.15.32:0/1767857792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090101380 0x7fd090103770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 --2- 172.21.15.32:0/1767857792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090103cb0 0x7fd0901060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 -- 172.21.15.32:0/1767857792 >> 172.21.15.32:0/1767857792 conn(0x7fd0900fad50 msgr2=0x7fd0900fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:54.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 -- 172.21.15.32:0/1767857792 shutdown_connections 2023-08-11T15:45:54.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.125+0000 7fd0963ff700 1 -- 172.21.15.32:0/1767857792 wait complete. 2023-08-11T15:45:54.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.126+0000 7fd0963ff700 1 Processor -- start 2023-08-11T15:45:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.126+0000 7fd0963ff700 1 -- start start 2023-08-11T15:45:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd0963ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090101380 0x7fd0901987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd0963ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090103cb0 0x7fd090198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd0963ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0901065e0 0x7fd09019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd0963ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0901a0370 con 0x7fd090101380 2023-08-11T15:45:54.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd0963ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0901a04b0 con 0x7fd0901065e0 2023-08-11T15:45:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd0963ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0901a0640 con 0x7fd090103cb0 2023-08-11T15:45:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090101380 0x7fd0901987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090101380 0x7fd0901987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38822/0 (socket says 172.21.15.32:38822) 2023-08-11T15:45:54.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 -- 172.21.15.32:0/397456711 learned_addr learned my addr 172.21.15.32:0/397456711 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:54.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd087fff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090103cb0 0x7fd090198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd09499c700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0901065e0 0x7fd09019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 -- 172.21.15.32:0/397456711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090103cb0 msgr2=0x7fd090198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090103cb0 0x7fd090198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 -- 172.21.15.32:0/397456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0901065e0 msgr2=0x7fd09019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0901065e0 0x7fd09019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.127+0000 7fd08ffff700 1 -- 172.21.15.32:0/397456711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0800095f0 con 0x7fd090101380 2023-08-11T15:45:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.128+0000 7fd08ffff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090101380 0x7fd0901987f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b680 tx=0x7fd0800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.128+0000 7fd08dffb700 1 -- 172.21.15.32:0/397456711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08001d070 con 0x7fd090101380 2023-08-11T15:45:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.128+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0901a08c0 con 0x7fd090101380 2023-08-11T15:45:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.128+0000 7fd08dffb700 1 -- 172.21.15.32:0/397456711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd080022470 con 0x7fd090101380 2023-08-11T15:45:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.128+0000 7fd08dffb700 1 -- 172.21.15.32:0/397456711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080017650 con 0x7fd090101380 2023-08-11T15:45:54.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.128+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0901a0d40 con 0x7fd090101380 2023-08-11T15:45:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.129+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0901081b0 con 0x7fd090101380 2023-08-11T15:45:54.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.131+0000 7fd08dffb700 1 -- 172.21.15.32:0/397456711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd0800225e0 con 0x7fd090101380 2023-08-11T15:45:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.133+0000 7fd08dffb700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd07c0a00a0 0x7fd07c0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.133+0000 7fd08dffb700 1 -- 172.21.15.32:0/397456711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd0800ccbb0 con 0x7fd090101380 2023-08-11T15:45:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.134+0000 7fd087fff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd07c0a00a0 0x7fd07c0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.134+0000 7fd087fff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd07c0a00a0 0x7fd07c0a2560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd078009c60 tx=0x7fd078009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:54.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.137+0000 7fd08dffb700 1 -- 172.21.15.32:0/397456711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd080096500 con 0x7fd090101380 2023-08-11T15:45:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.352+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd09010c350 con 0x7fd090101380 2023-08-11T15:45:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.353+0000 7fd08dffb700 1 -- 172.21.15.32:0/397456711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24460 (secure 0 0 0) 0x7fd080099b20 con 0x7fd090101380 2023-08-11T15:45:54.356 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:54.356 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:44:55.185701+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:45:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd07c0a00a0 msgr2=0x7fd07c0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd07c0a00a0 0x7fd07c0a2560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd078009c60 tx=0x7fd078009260 comp rx=0 tx=0).stop 2023-08-11T15:45:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090101380 msgr2=0x7fd0901987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090101380 0x7fd0901987f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b680 tx=0x7fd0800057b0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 shutdown_connections 2023-08-11T15:45:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd07c0a00a0 0x7fd07c0a2560 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd090101380 0x7fd0901987f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0901065e0 0x7fd09019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 --2- 172.21.15.32:0/397456711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd090103cb0 0x7fd090198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 >> 172.21.15.32:0/397456711 conn(0x7fd0900fad50 msgr2=0x7fd090101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 shutdown_connections 2023-08-11T15:45:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.356+0000 7fd0963ff700 1 -- 172.21.15.32:0/397456711 wait complete. 2023-08-11T15:45:54.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:45:54.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:45:54.439 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:54 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3890471773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:45:54.440 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:54 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1007390722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:54.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:54 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3890471773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:45:54.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:54 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1007390722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:54 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3890471773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:45:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:54 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1007390722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 -- 172.21.15.32:0/4253416313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22581024a0 msgr2=0x7f2258108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 --2- 172.21.15.32:0/4253416313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22581024a0 0x7f2258108d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f225000b0a0 tx=0x7f225000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 -- 172.21.15.32:0/4253416313 shutdown_connections 2023-08-11T15:45:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 --2- 172.21.15.32:0/4253416313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22581024a0 0x7f2258108d50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 --2- 172.21.15.32:0/4253416313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2258100910 0x7f2258100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 --2- 172.21.15.32:0/4253416313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258101b00 0x7f2258101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 -- 172.21.15.32:0/4253416313 >> 172.21.15.32:0/4253416313 conn(0x7f22580fbe60 msgr2=0x7f22580fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 -- 172.21.15.32:0/4253416313 shutdown_connections 2023-08-11T15:45:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.583+0000 7f225e3ff700 1 -- 172.21.15.32:0/4253416313 wait complete. 2023-08-11T15:45:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 Processor -- start 2023-08-11T15:45:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 -- start start 2023-08-11T15:45:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258100910 0x7f2258196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2258101b00 0x7f2258196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581024a0 0x7f225819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225819e2b0 con 0x7f2258100910 2023-08-11T15:45:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225819e3f0 con 0x7f22581024a0 2023-08-11T15:45:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.584+0000 7f225e3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225819e580 con 0x7f2258101b00 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258100910 0x7f2258196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258100910 0x7f2258196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41894/0 (socket says 172.21.15.32:41894) 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 -- 172.21.15.32:0/3321537505 learned_addr learned my addr 172.21.15.32:0/3321537505 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f225c99c700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581024a0 0x7f225819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f22577fe700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2258101b00 0x7f2258196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 -- 172.21.15.32:0/3321537505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2258101b00 msgr2=0x7f2258196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2258101b00 0x7f2258196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 -- 172.21.15.32:0/3321537505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581024a0 msgr2=0x7f225819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581024a0 0x7f225819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 -- 172.21.15.32:0/3321537505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c009520 con 0x7f2258100910 2023-08-11T15:45:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f225c99c700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581024a0 0x7f225819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:45:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f22577fe700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2258101b00 0x7f2258196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:45:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.585+0000 7f2257fff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258100910 0x7f2258196730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f224800e970 tx=0x7f224800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.586+0000 7f22557fa700 1 -- 172.21.15.32:0/3321537505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248009790 con 0x7f2258100910 2023-08-11T15:45:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.586+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2250009d00 con 0x7f2258100910 2023-08-11T15:45:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.586+0000 7f22557fa700 1 -- 172.21.15.32:0/3321537505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22480044e0 con 0x7f2258100910 2023-08-11T15:45:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.586+0000 7f22557fa700 1 -- 172.21.15.32:0/3321537505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248003e30 con 0x7f2258100910 2023-08-11T15:45:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.586+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225819ec30 con 0x7f2258100910 2023-08-11T15:45:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.587+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225810c180 con 0x7f2258100910 2023-08-11T15:45:54.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.589+0000 7f22557fa700 1 -- 172.21.15.32:0/3321537505 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f2248004000 con 0x7f2258100910 2023-08-11T15:45:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.590+0000 7f22557fa700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f224009ffe0 0x7f22400a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.590+0000 7f22557fa700 1 -- 172.21.15.32:0/3321537505 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2248014070 con 0x7f2258100910 2023-08-11T15:45:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.590+0000 7f22577fe700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f224009ffe0 0x7f22400a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.591+0000 7f22577fe700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f224009ffe0 0x7f22400a24a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f224c009e40 tx=0x7f224c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.594+0000 7f22557fa700 1 -- 172.21.15.32:0/3321537505 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f22480952f0 con 0x7f2258100910 2023-08-11T15:45:54.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.803+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22581077f0 con 0x7f2258100910 2023-08-11T15:45:54.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.803+0000 7f22557fa700 1 -- 172.21.15.32:0/3321537505 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2248098910 con 0x7f2258100910 2023-08-11T15:45:54.806 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:45:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f224009ffe0 msgr2=0x7f22400a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f224009ffe0 0x7f22400a24a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f224c009e40 tx=0x7f224c009260 comp rx=0 tx=0).stop 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258100910 msgr2=0x7f2258196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258100910 0x7f2258196730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f224800e970 tx=0x7f224800ed30 comp rx=0 tx=0).stop 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 shutdown_connections 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f224009ffe0 0x7f22400a24a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2258100910 0x7f2258196730 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581024a0 0x7f225819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 --2- 172.21.15.32:0/3321537505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2258101b00 0x7f2258196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 >> 172.21.15.32:0/3321537505 conn(0x7f22580fbe60 msgr2=0x7f22580fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 shutdown_connections 2023-08-11T15:45:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:54.806+0000 7f225e3ff700 1 -- 172.21.15.32:0/3321537505 wait complete. 2023-08-11T15:45:54.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:45:55.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 -- 172.21.15.32:0/4210485490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 msgr2=0x7f6d94108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 --2- 172.21.15.32:0/4210485490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d94108d50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b0d0 tx=0x7f6d8c00b3e0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 -- 172.21.15.32:0/4210485490 shutdown_connections 2023-08-11T15:45:55.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 --2- 172.21.15.32:0/4210485490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d94108d50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 --2- 172.21.15.32:0/4210485490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d94100910 0x7f6d94100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 --2- 172.21.15.32:0/4210485490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d94101b00 0x7f6d94101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 -- 172.21.15.32:0/4210485490 >> 172.21.15.32:0/4210485490 conn(0x7f6d940fbe60 msgr2=0x7f6d940fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:55.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 -- 172.21.15.32:0/4210485490 shutdown_connections 2023-08-11T15:45:55.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.033+0000 7f6d991ff700 1 -- 172.21.15.32:0/4210485490 wait complete. 2023-08-11T15:45:55.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 Processor -- start 2023-08-11T15:45:55.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 -- start start 2023-08-11T15:45:55.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d94100910 0x7f6d94198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d94101b00 0x7f6d94198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d9419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d941a04b0 con 0x7f6d941024a0 2023-08-11T15:45:55.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d941a05f0 con 0x7f6d94101b00 2023-08-11T15:45:55.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d991ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d941a0780 con 0x7f6d94100910 2023-08-11T15:45:55.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d9419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d9419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41916/0 (socket says 172.21.15.32:41916) 2023-08-11T15:45:55.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 -- 172.21.15.32:0/2628777206 learned_addr learned my addr 172.21.15.32:0/2628777206 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d92d9d700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d94100910 0x7f6d94198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9259c700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d94101b00 0x7f6d94198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 -- 172.21.15.32:0/2628777206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d94100910 msgr2=0x7f6d94198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d94100910 0x7f6d94198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 -- 172.21.15.32:0/2628777206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d94101b00 msgr2=0x7f6d94198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d94101b00 0x7f6d94198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 -- 172.21.15.32:0/2628777206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c009d00 con 0x7f6d941024a0 2023-08-11T15:45:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.035+0000 7f6d9359e700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d9419fee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c008eb0 tx=0x7f6d8c008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:55.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.036+0000 7f6d7bfff700 1 -- 172.21.15.32:0/2628777206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00e040 con 0x7f6d941024a0 2023-08-11T15:45:55.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.036+0000 7f6d7bfff700 1 -- 172.21.15.32:0/2628777206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c00bdb0 con 0x7f6d941024a0 2023-08-11T15:45:55.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.036+0000 7f6d7bfff700 1 -- 172.21.15.32:0/2628777206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c01d470 con 0x7f6d941024a0 2023-08-11T15:45:55.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.036+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d941a0a00 con 0x7f6d941024a0 2023-08-11T15:45:55.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.036+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d941a0e30 con 0x7f6d941024a0 2023-08-11T15:45:55.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.037+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9410c180 con 0x7f6d941024a0 2023-08-11T15:45:55.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.039+0000 7f6d7bfff700 1 -- 172.21.15.32:0/2628777206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f6d8c005020 con 0x7f6d941024a0 2023-08-11T15:45:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.040+0000 7f6d7bfff700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6d7c0a0060 0x7f6d7c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.040+0000 7f6d7bfff700 1 -- 172.21.15.32:0/2628777206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6d8c0cc5f0 con 0x7f6d941024a0 2023-08-11T15:45:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.040+0000 7f6d92d9d700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6d7c0a0060 0x7f6d7c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.041+0000 7f6d92d9d700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6d7c0a0060 0x7f6d7c0a2520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d94101960 tx=0x7f6d84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.044+0000 7f6d7bfff700 1 -- 172.21.15.32:0/2628777206 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6d8c095f40 con 0x7f6d941024a0 2023-08-11T15:45:55.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.264+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d941a1050 con 0x7f6d941024a0 2023-08-11T15:45:55.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.265+0000 7f6d7bfff700 1 -- 172.21.15.32:0/2628777206 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d8c099560 con 0x7f6d941024a0 2023-08-11T15:45:55.267 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:45:55.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6d7c0a0060 msgr2=0x7f6d7c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6d7c0a0060 0x7f6d7c0a2520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d94101960 tx=0x7f6d84009260 comp rx=0 tx=0).stop 2023-08-11T15:45:55.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 msgr2=0x7f6d9419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d9419fee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c008eb0 tx=0x7f6d8c008ee0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 shutdown_connections 2023-08-11T15:45:55.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f6d7c0a0060 0x7f6d7c0a2520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d941024a0 0x7f6d9419fee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d94100910 0x7f6d94198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 --2- 172.21.15.32:0/2628777206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d94101b00 0x7f6d94198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 >> 172.21.15.32:0/2628777206 conn(0x7f6d940fbe60 msgr2=0x7f6d940fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:55.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.267+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 shutdown_connections 2023-08-11T15:45:55.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.268+0000 7f6d991ff700 1 -- 172.21.15.32:0/2628777206 wait complete. 2023-08-11T15:45:55.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:45:55.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:55 smithi134 ceph-mon[130447]: pgmap v320: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:55.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:55 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-08-11T15:45:55.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:55 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/397456711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:55.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:55 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3321537505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:45:55.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/254212784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e81024a0 msgr2=0x7ff4e8108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/254212784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e81024a0 0x7ff4e8108d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b0d0 tx=0x7ff4e000b3e0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/254212784 shutdown_connections 2023-08-11T15:45:55.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/254212784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e81024a0 0x7ff4e8108d50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/254212784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e8100910 0x7ff4e8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/254212784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e8101b00 0x7ff4e8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/254212784 >> 172.21.15.32:0/254212784 conn(0x7ff4e80fbe60 msgr2=0x7ff4e80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:55.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/254212784 shutdown_connections 2023-08-11T15:45:55.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/254212784 wait complete. 2023-08-11T15:45:55.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 Processor -- start 2023-08-11T15:45:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- start start 2023-08-11T15:45:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e8100910 0x7ff4e80746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e8101b00 0x7ff4e8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e81024a0 0x7ff4e8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80739f0 con 0x7ff4e8100910 2023-08-11T15:45:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8073b60 con 0x7ff4e81024a0 2023-08-11T15:45:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.498+0000 7ff4ef7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e81a0780 con 0x7ff4e8101b00 2023-08-11T15:45:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e8100910 0x7ff4e80746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e8100910 0x7ff4e80746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41938/0 (socket says 172.21.15.32:41938) 2023-08-11T15:45:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 -- 172.21.15.32:0/2205002839 learned_addr learned my addr 172.21.15.32:0/2205002839 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4edd9c700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e81024a0 0x7ff4e8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ecd9a700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e8101b00 0x7ff4e8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 -- 172.21.15.32:0/2205002839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e8101b00 msgr2=0x7ff4e8072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e8101b00 0x7ff4e8072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 -- 172.21.15.32:0/2205002839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e81024a0 msgr2=0x7ff4e8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e81024a0 0x7ff4e8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.499+0000 7ff4ed59b700 1 -- 172.21.15.32:0/2205002839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0009d00 con 0x7ff4e8100910 2023-08-11T15:45:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.500+0000 7ff4ed59b700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e8100910 0x7ff4e80746d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400d980 tx=0x7ff4e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.500+0000 7ff4de7fc700 1 -- 172.21.15.32:0/2205002839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4004020 con 0x7ff4e8100910 2023-08-11T15:45:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.500+0000 7ff4de7fc700 1 -- 172.21.15.32:0/2205002839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e400f690 con 0x7ff4e8100910 2023-08-11T15:45:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.500+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e81a0a60 con 0x7ff4e8100910 2023-08-11T15:45:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.500+0000 7ff4de7fc700 1 -- 172.21.15.32:0/2205002839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4003a60 con 0x7ff4e8100910 2023-08-11T15:45:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.500+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e81a0f70 con 0x7ff4e8100910 2023-08-11T15:45:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.501+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e8107e60 con 0x7ff4e8100910 2023-08-11T15:45:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.504+0000 7ff4de7fc700 1 -- 172.21.15.32:0/2205002839 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff4e400fb00 con 0x7ff4e8100910 2023-08-11T15:45:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.506+0000 7ff4de7fc700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff4d409ffe0 0x7ff4d40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.506+0000 7ff4de7fc700 1 -- 172.21.15.32:0/2205002839 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff4e40cb2f0 con 0x7ff4e8100910 2023-08-11T15:45:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.506+0000 7ff4ecd9a700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff4d409ffe0 0x7ff4d40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.506+0000 7ff4ecd9a700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff4d409ffe0 0x7ff4d40a24a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8005950 tx=0x7ff4d800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.511+0000 7ff4de7fc700 1 -- 172.21.15.32:0/2205002839 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff4e4094b90 con 0x7ff4e8100910 2023-08-11T15:45:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:55 smithi032 ceph-mon[97103]: pgmap v320: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:55 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-08-11T15:45:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:55 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/397456711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:55.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:55 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3321537505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:45:55.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:55 smithi192 ceph-mon[130806]: pgmap v320: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:45:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:55 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-08-11T15:45:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:55 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/397456711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:55 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3321537505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:45:55.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.723+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4e810c3f0 con 0x7ff4e8100910 2023-08-11T15:45:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.723+0000 7ff4de7fc700 1 -- 172.21.15.32:0/2205002839 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4e40981b0 con 0x7ff4e8100910 2023-08-11T15:45:55.725 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:45:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.725+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff4d409ffe0 msgr2=0x7ff4d40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.725+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff4d409ffe0 0x7ff4d40a24a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8005950 tx=0x7ff4d800b410 comp rx=0 tx=0).stop 2023-08-11T15:45:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e8100910 msgr2=0x7ff4e80746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e8100910 0x7ff4e80746d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400d980 tx=0x7ff4e400dc90 comp rx=0 tx=0).stop 2023-08-11T15:45:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 shutdown_connections 2023-08-11T15:45:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff4d409ffe0 0x7ff4d40a24a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4e8100910 0x7ff4e80746d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e81024a0 0x7ff4e8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 --2- 172.21.15.32:0/2205002839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e8101b00 0x7ff4e8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 >> 172.21.15.32:0/2205002839 conn(0x7ff4e80fbe60 msgr2=0x7ff4e8104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 shutdown_connections 2023-08-11T15:45:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.726+0000 7ff4ef7ff700 1 -- 172.21.15.32:0/2205002839 wait complete. 2023-08-11T15:45:55.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:45:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 -- 172.21.15.32:0/2518491769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 msgr2=0x7f52e8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 --2- 172.21.15.32:0/2518491769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e8100d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b0a0 tx=0x7f52dc00b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 -- 172.21.15.32:0/2518491769 shutdown_connections 2023-08-11T15:45:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 --2- 172.21.15.32:0/2518491769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e81024c0 0x7f52e8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 --2- 172.21.15.32:0/2518491769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e8100d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 --2- 172.21.15.32:0/2518491769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e8101b20 0x7f52e8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 -- 172.21.15.32:0/2518491769 >> 172.21.15.32:0/2518491769 conn(0x7f52e80fbea0 msgr2=0x7f52e80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 -- 172.21.15.32:0/2518491769 shutdown_connections 2023-08-11T15:45:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.956+0000 7f52eedff700 1 -- 172.21.15.32:0/2518491769 wait complete. 2023-08-11T15:45:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.957+0000 7f52eedff700 1 Processor -- start 2023-08-11T15:45:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.957+0000 7f52eedff700 1 -- start start 2023-08-11T15:45:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52eedff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52eedff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8101b20 0x7f52e8198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52eedff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e81024c0 0x7f52e819ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52eedff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81a0540 con 0x7f52e8101b20 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52eedff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81a0680 con 0x7f52e81024c0 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52eedff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81a0810 con 0x7f52e8100930 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e81989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:45388/0 (socket says 172.21.15.32:45388) 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 -- 172.21.15.32:0/3913166603 learned_addr learned my addr 172.21.15.32:0/3913166603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8101b20 0x7f52e8198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ed39c700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e81024c0 0x7f52e819ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 -- 172.21.15.32:0/3913166603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e81024c0 msgr2=0x7f52e819ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e81024c0 0x7f52e819ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 -- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8101b20 msgr2=0x7f52e8198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8101b20 0x7f52e8198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.958+0000 7f52ecb9b700 1 -- 172.21.15.32:0/3913166603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc009d00 con 0x7f52e8100930 2023-08-11T15:45:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.959+0000 7f52ecb9b700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e81989c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b780 tx=0x7f52dc00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.959+0000 7f52e5ffb700 1 -- 172.21.15.32:0/3913166603 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00e040 con 0x7f52e8100930 2023-08-11T15:45:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.959+0000 7f52e5ffb700 1 -- 172.21.15.32:0/3913166603 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52dc004e60 con 0x7f52e8100930 2023-08-11T15:45:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.959+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e81a0a90 con 0x7f52e8100930 2023-08-11T15:45:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.959+0000 7f52e5ffb700 1 -- 172.21.15.32:0/3913166603 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc0052a0 con 0x7f52e8100930 2023-08-11T15:45:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.959+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e81a0f10 con 0x7f52e8100930 2023-08-11T15:45:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.960+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e8192f80 con 0x7f52e8100930 2023-08-11T15:45:55.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.962+0000 7f52e5ffb700 1 -- 172.21.15.32:0/3913166603 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f52dc01d470 con 0x7f52e8100930 2023-08-11T15:45:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.963+0000 7f52e5ffb700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d809ffe0 0x7f52d80a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.964+0000 7f52e5ffb700 1 -- 172.21.15.32:0/3913166603 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f52dc0cd450 con 0x7f52e8100930 2023-08-11T15:45:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.964+0000 7f52e7fff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d809ffe0 0x7f52d80a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.964+0000 7f52e7fff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d809ffe0 0x7f52d80a24a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009c60 tx=0x7f52d4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:55.966+0000 7f52e5ffb700 1 -- 172.21.15.32:0/3913166603 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f52dc096e10 con 0x7f52e8100930 2023-08-11T15:45:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.176+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f52e810c320 con 0x7f52e8100930 2023-08-11T15:45:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.177+0000 7f52e5ffb700 1 -- 172.21.15.32:0/3913166603 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f52dc09a430 con 0x7f52e8100930 2023-08-11T15:45:56.178 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:45:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.179+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d809ffe0 msgr2=0x7f52d80a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.179+0000 7f52eedff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d809ffe0 0x7f52d80a24a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009c60 tx=0x7f52d4009260 comp rx=0 tx=0).stop 2023-08-11T15:45:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 msgr2=0x7f52e81989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e81989c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b780 tx=0x7f52dc00bd60 comp rx=0 tx=0).stop 2023-08-11T15:45:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 shutdown_connections 2023-08-11T15:45:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f52d809ffe0 0x7f52d80a24a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e81024c0 0x7f52e819ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52e8100930 0x7f52e81989c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 --2- 172.21.15.32:0/3913166603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8101b20 0x7f52e8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 >> 172.21.15.32:0/3913166603 conn(0x7f52e80fbea0 msgr2=0x7f52e8107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:56.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 shutdown_connections 2023-08-11T15:45:56.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.180+0000 7f52eedff700 1 -- 172.21.15.32:0/3913166603 wait complete. 2023-08-11T15:45:56.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:45:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.405+0000 7f2926bff700 1 -- 172.21.15.32:0/418389302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29201087f0 msgr2=0x7f292010ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.405+0000 7f2926bff700 1 --2- 172.21.15.32:0/418389302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29201087f0 0x7f292010ac80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f291800b0a0 tx=0x7f291800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.406+0000 7f2926bff700 1 -- 172.21.15.32:0/418389302 shutdown_connections 2023-08-11T15:45:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.406+0000 7f2926bff700 1 --2- 172.21.15.32:0/418389302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29201087f0 0x7f292010ac80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.406+0000 7f2926bff700 1 --2- 172.21.15.32:0/418389302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920074960 0x7f2920072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.406+0000 7f2926bff700 1 --2- 172.21.15.32:0/418389302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2920073380 0x7f2920073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.406+0000 7f2926bff700 1 -- 172.21.15.32:0/418389302 >> 172.21.15.32:0/418389302 conn(0x7f29200fbe60 msgr2=0x7f29200fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.406+0000 7f2926bff700 1 -- 172.21.15.32:0/418389302 shutdown_connections 2023-08-11T15:45:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.406+0000 7f2926bff700 1 -- 172.21.15.32:0/418389302 wait complete. 2023-08-11T15:45:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 Processor -- start 2023-08-11T15:45:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 -- start start 2023-08-11T15:45:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920073380 0x7f29201988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920074960 0x7f2920198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29201087f0 0x7f292019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201a0440 con 0x7f2920073380 2023-08-11T15:45:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201a0580 con 0x7f2920074960 2023-08-11T15:45:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.407+0000 7f2926bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201a0710 con 0x7f29201087f0 2023-08-11T15:45:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920073380 0x7f29201988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920073380 0x7f29201988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41976/0 (socket says 172.21.15.32:41976) 2023-08-11T15:45:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 -- 172.21.15.32:0/3403889874 learned_addr learned my addr 172.21.15.32:0/3403889874 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f291ffff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920074960 0x7f2920198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292519c700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29201087f0 0x7f292019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 -- 172.21.15.32:0/3403889874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29201087f0 msgr2=0x7f292019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29201087f0 0x7f292019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 -- 172.21.15.32:0/3403889874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920074960 msgr2=0x7f2920198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920074960 0x7f2920198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 -- 172.21.15.32:0/3403889874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918009d00 con 0x7f2920073380 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.408+0000 7f292499b700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920073380 0x7f29201988c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f291400e9a0 tx=0x7f291400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.409+0000 7f291dffb700 1 -- 172.21.15.32:0/3403889874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914009790 con 0x7f2920073380 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.409+0000 7f291dffb700 1 -- 172.21.15.32:0/3403889874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2914010430 con 0x7f2920073380 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.409+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29201a09f0 con 0x7f2920073380 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.409+0000 7f291dffb700 1 -- 172.21.15.32:0/3403889874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914018680 con 0x7f2920073380 2023-08-11T15:45:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.409+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29201a0f00 con 0x7f2920073380 2023-08-11T15:45:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.410+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920106ea0 con 0x7f2920073380 2023-08-11T15:45:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.412+0000 7f291dffb700 1 -- 172.21.15.32:0/3403889874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f29140105a0 con 0x7f2920073380 2023-08-11T15:45:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.414+0000 7f291dffb700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f29100a0020 0x7f29100a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.414+0000 7f291dffb700 1 -- 172.21.15.32:0/3403889874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2914014070 con 0x7f2920073380 2023-08-11T15:45:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.414+0000 7f291ffff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f29100a0020 0x7f29100a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.415+0000 7f291ffff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f29100a0020 0x7f29100a24e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2920074680 tx=0x7f290c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.418+0000 7f291dffb700 1 -- 172.21.15.32:0/3403889874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2914051c30 con 0x7f2920073380 2023-08-11T15:45:56.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:56 smithi134 ceph-mon[130447]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-08-11T15:45:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:56 smithi134 ceph-mon[130447]: osdmap e91: 12 total, 12 up, 12 in 2023-08-11T15:45:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:56 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2628777206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:56 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2205002839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:56 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3913166603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:56 smithi032 ceph-mon[97103]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-08-11T15:45:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:56 smithi032 ceph-mon[97103]: osdmap e91: 12 total, 12 up, 12 in 2023-08-11T15:45:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:56 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2628777206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:56 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2205002839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:56 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3913166603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:56 smithi192 ceph-mon[130806]: from='mgr.14148 172.21.15.32:0/695122276' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-08-11T15:45:56.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:56 smithi192 ceph-mon[130806]: osdmap e91: 12 total, 12 up, 12 in 2023-08-11T15:45:56.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:56 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2628777206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:56 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2205002839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:56 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3913166603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:45:56.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.676+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f292010e370 con 0x7f2920073380 2023-08-11T15:45:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.677+0000 7f291dffb700 1 -- 172.21.15.32:0/3403889874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f29140951c0 con 0x7f2920073380 2023-08-11T15:45:56.679 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:56.679 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:52.172791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:45:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.679+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f29100a0020 msgr2=0x7f29100a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.679+0000 7f2926bff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f29100a0020 0x7f29100a24e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2920074680 tx=0x7f290c009260 comp rx=0 tx=0).stop 2023-08-11T15:45:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920073380 msgr2=0x7f29201988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920073380 0x7f29201988c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f291400e9a0 tx=0x7f291400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 shutdown_connections 2023-08-11T15:45:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f29100a0020 0x7f29100a24e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920073380 0x7f29201988c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29201087f0 0x7f292019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 --2- 172.21.15.32:0/3403889874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920074960 0x7f2920198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 >> 172.21.15.32:0/3403889874 conn(0x7f29200fbe60 msgr2=0x7f29200fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 shutdown_connections 2023-08-11T15:45:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.680+0000 7f2926bff700 1 -- 172.21.15.32:0/3403889874 wait complete. 2023-08-11T15:45:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-08-11T15:45:56.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:45:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 -- 172.21.15.32:0/1420519719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c100790 msgr2=0x7fc84c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 --2- 172.21.15.32:0/1420519719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c100790 0x7fc84c100bb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc8480099c0 tx=0x7fc848009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 -- 172.21.15.32:0/1420519719 shutdown_connections 2023-08-11T15:45:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 --2- 172.21.15.32:0/1420519719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c100790 0x7fc84c100bb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 --2- 172.21.15.32:0/1420519719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc84c102320 0x7fc84c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 --2- 172.21.15.32:0/1420519719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c101980 0x7fc84c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 -- 172.21.15.32:0/1420519719 >> 172.21.15.32:0/1420519719 conn(0x7fc84c0fbce0 msgr2=0x7fc84c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 -- 172.21.15.32:0/1420519719 shutdown_connections 2023-08-11T15:45:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.904+0000 7fc85239d700 1 -- 172.21.15.32:0/1420519719 wait complete. 2023-08-11T15:45:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 Processor -- start 2023-08-11T15:45:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 -- start start 2023-08-11T15:45:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc84c100790 0x7fc84c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c101980 0x7fc84c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c102320 0x7fc84c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0739f0 con 0x7fc84c102320 2023-08-11T15:45:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c073b60 con 0x7fc84c100790 2023-08-11T15:45:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.905+0000 7fc85239d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1a03c0 con 0x7fc84c101980 2023-08-11T15:45:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c102320 0x7fc84c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c102320 0x7fc84c073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41986/0 (socket says 172.21.15.32:41986) 2023-08-11T15:45:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 -- 172.21.15.32:0/914315070 learned_addr learned my addr 172.21.15.32:0/914315070 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc85139b700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc84c100790 0x7fc84c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc850b9a700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c101980 0x7fc84c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 -- 172.21.15.32:0/914315070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c101980 msgr2=0x7fc84c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c101980 0x7fc84c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 -- 172.21.15.32:0/914315070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc84c100790 msgr2=0x7fc84c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc84c100790 0x7fc84c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 -- 172.21.15.32:0/914315070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8480095f0 con 0x7fc84c102320 2023-08-11T15:45:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.906+0000 7fc851b9c700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c102320 0x7fc84c073250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc84400ef30 tx=0x7fc84400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.907+0000 7fc8427fc700 1 -- 172.21.15.32:0/914315070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844018070 con 0x7fc84c102320 2023-08-11T15:45:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.907+0000 7fc8427fc700 1 -- 172.21.15.32:0/914315070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84400f040 con 0x7fc84c102320 2023-08-11T15:45:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.907+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c1a0640 con 0x7fc84c102320 2023-08-11T15:45:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.907+0000 7fc8427fc700 1 -- 172.21.15.32:0/914315070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84400cd00 con 0x7fc84c102320 2023-08-11T15:45:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.907+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c1a0b00 con 0x7fc84c102320 2023-08-11T15:45:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.908+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c107c30 con 0x7fc84c102320 2023-08-11T15:45:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.910+0000 7fc8427fc700 1 -- 172.21.15.32:0/914315070 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc844003a00 con 0x7fc84c102320 2023-08-11T15:45:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.911+0000 7fc8427fc700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8380a00e0 0x7fc8380a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.911+0000 7fc8427fc700 1 -- 172.21.15.32:0/914315070 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc8440cb240 con 0x7fc84c102320 2023-08-11T15:45:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.912+0000 7fc85139b700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8380a00e0 0x7fc8380a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.912+0000 7fc85139b700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8380a00e0 0x7fc8380a25a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc848009990 tx=0x7fc848005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:56.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:56.916+0000 7fc8427fc700 1 -- 172.21.15.32:0/914315070 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc844094b70 con 0x7fc84c102320 2023-08-11T15:45:57.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.126+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc84c10c000 con 0x7fc84c102320 2023-08-11T15:45:57.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.128+0000 7fc8427fc700 1 -- 172.21.15.32:0/914315070 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24508 (secure 0 0 0) 0x7fc844098190 con 0x7fc84c102320 2023-08-11T15:45:57.130 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:57.130 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:45:56.189759+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:45:57.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8380a00e0 msgr2=0x7fc8380a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8380a00e0 0x7fc8380a25a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc848009990 tx=0x7fc848005fb0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c102320 msgr2=0x7fc84c073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c102320 0x7fc84c073250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc84400ef30 tx=0x7fc84400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 shutdown_connections 2023-08-11T15:45:57.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc8380a00e0 0x7fc8380a25a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c102320 0x7fc84c073250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.130+0000 7fc85239d700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc84c100790 0x7fc84c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.131+0000 7fc85239d700 1 --2- 172.21.15.32:0/914315070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c101980 0x7fc84c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.131+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 >> 172.21.15.32:0/914315070 conn(0x7fc84c0fbce0 msgr2=0x7fc84c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.131+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 shutdown_connections 2023-08-11T15:45:57.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.131+0000 7fc85239d700 1 -- 172.21.15.32:0/914315070 wait complete. 2023-08-11T15:45:57.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:45:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.355+0000 7fb9205ff700 1 -- 172.21.15.32:0/40103391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 msgr2=0x7fb918072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.355+0000 7fb9205ff700 1 --2- 172.21.15.32:0/40103391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918072db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb914009990 tx=0x7fb914009ca0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.356+0000 7fb9205ff700 1 -- 172.21.15.32:0/40103391 shutdown_connections 2023-08-11T15:45:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.356+0000 7fb9205ff700 1 --2- 172.21.15.32:0/40103391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918072db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.356+0000 7fb9205ff700 1 --2- 172.21.15.32:0/40103391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9181087f0 0x7fb91810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.356+0000 7fb9205ff700 1 --2- 172.21.15.32:0/40103391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb918073380 0x7fb918073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.356+0000 7fb9205ff700 1 -- 172.21.15.32:0/40103391 >> 172.21.15.32:0/40103391 conn(0x7fb9180fbe60 msgr2=0x7fb9180fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.356+0000 7fb9205ff700 1 -- 172.21.15.32:0/40103391 shutdown_connections 2023-08-11T15:45:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.356+0000 7fb9205ff700 1 -- 172.21.15.32:0/40103391 wait complete. 2023-08-11T15:45:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 Processor -- start 2023-08-11T15:45:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 -- start start 2023-08-11T15:45:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb918073380 0x7fb918198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9181087f0 0x7fb91819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181a0390 con 0x7fb918074960 2023-08-11T15:45:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181a04d0 con 0x7fb918073380 2023-08-11T15:45:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.357+0000 7fb9205ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181a0660 con 0x7fb9181087f0 2023-08-11T15:45:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42000/0 (socket says 172.21.15.32:42000) 2023-08-11T15:45:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 -- 172.21.15.32:0/1835667322 learned_addr learned my addr 172.21.15.32:0/1835667322 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91eb9c700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9181087f0 0x7fb91819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91e39b700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb918073380 0x7fb918198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 -- 172.21.15.32:0/1835667322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9181087f0 msgr2=0x7fb91819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9181087f0 0x7fb91819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 -- 172.21.15.32:0/1835667322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb918073380 msgr2=0x7fb918198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb918073380 0x7fb918198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 -- 172.21.15.32:0/1835667322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140095f0 con 0x7fb918074960 2023-08-11T15:45:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91e39b700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb918073380 0x7fb918198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb91db9a700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918198d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d980 tx=0x7fb90800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.358+0000 7fb90f7fe700 1 -- 172.21.15.32:0/1835667322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90800f890 con 0x7fb918074960 2023-08-11T15:45:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.359+0000 7fb90f7fe700 1 -- 172.21.15.32:0/1835667322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb908010460 con 0x7fb918074960 2023-08-11T15:45:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.359+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9181a0940 con 0x7fb918074960 2023-08-11T15:45:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.359+0000 7fb90f7fe700 1 -- 172.21.15.32:0/1835667322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080195c0 con 0x7fb918074960 2023-08-11T15:45:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.359+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9181a0ea0 con 0x7fb918074960 2023-08-11T15:45:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.360+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb918106ea0 con 0x7fb918074960 2023-08-11T15:45:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.362+0000 7fb90f7fe700 1 -- 172.21.15.32:0/1835667322 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fb9080105d0 con 0x7fb918074960 2023-08-11T15:45:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.363+0000 7fb90f7fe700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb9040a00a0 0x7fb9040a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.363+0000 7fb90f7fe700 1 -- 172.21.15.32:0/1835667322 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb9080cb440 con 0x7fb918074960 2023-08-11T15:45:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.363+0000 7fb91e39b700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb9040a00a0 0x7fb9040a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.364+0000 7fb91e39b700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb9040a00a0 0x7fb9040a2560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb91400b5c0 tx=0x7fb914005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.367+0000 7fb90f7fe700 1 -- 172.21.15.32:0/1835667322 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb908094ce0 con 0x7fb918074960 2023-08-11T15:45:57.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:57 smithi134 ceph-mon[130447]: pgmap v322: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:45:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:57 smithi134 ceph-mon[130447]: osdmap e92: 12 total, 12 up, 12 in 2023-08-11T15:45:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:57 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3403889874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:57.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:57 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/914315070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:57 smithi032 ceph-mon[97103]: pgmap v322: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:45:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:57 smithi032 ceph-mon[97103]: osdmap e92: 12 total, 12 up, 12 in 2023-08-11T15:45:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:57 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3403889874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:57.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:57 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/914315070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:57.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:57 smithi192 ceph-mon[130806]: pgmap v322: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:45:57.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:57 smithi192 ceph-mon[130806]: osdmap e92: 12 total, 12 up, 12 in 2023-08-11T15:45:57.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:57 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3403889874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:57.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:57 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/914315070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:45:57.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.622+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb91810e570 con 0x7fb918074960 2023-08-11T15:45:57.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.622+0000 7fb90f7fe700 1 -- 172.21.15.32:0/1835667322 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fb908098300 con 0x7fb918074960 2023-08-11T15:45:57.624 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:45:57.624 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:52.172791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:45:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb9040a00a0 msgr2=0x7fb9040a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb9040a00a0 0x7fb9040a2560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb91400b5c0 tx=0x7fb914005fd0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 msgr2=0x7fb918198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918198d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d980 tx=0x7fb90800dc90 comp rx=0 tx=0).stop 2023-08-11T15:45:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 shutdown_connections 2023-08-11T15:45:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fb9040a00a0 0x7fb9040a2560 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9181087f0 0x7fb91819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb918073380 0x7fb918198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 --2- 172.21.15.32:0/1835667322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918074960 0x7fb918198d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 >> 172.21.15.32:0/1835667322 conn(0x7fb9180fbe60 msgr2=0x7fb9180fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 shutdown_connections 2023-08-11T15:45:57.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.625+0000 7fb9205ff700 1 -- 172.21.15.32:0/1835667322 wait complete. 2023-08-11T15:45:57.628 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-08-11T15:45:57.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-11T15:44:19.707016+0000', 'modified': '2023-08-11T15:44:52.172791+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.32:6841/1060395589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6840', 'nonce': 1060395589}, {'type': 'v1', 'addr': '172.21.15.32:6841', 'nonce': 1060395589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/945169036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 945169036}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 945169036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-11T15:45:57.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-11T15:45:57.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T15:45:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 -- 172.21.15.32:0/229972227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4101b20 msgr2=0x7fade4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 --2- 172.21.15.32:0/229972227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4101b20 0x7fade4101f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fadd40099c0 tx=0x7fadd4009cd0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 -- 172.21.15.32:0/229972227 shutdown_connections 2023-08-11T15:45:57.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 --2- 172.21.15.32:0/229972227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 0x7fade4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 --2- 172.21.15.32:0/229972227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4100930 0x7fade4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 --2- 172.21.15.32:0/229972227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4101b20 0x7fade4101f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 -- 172.21.15.32:0/229972227 >> 172.21.15.32:0/229972227 conn(0x7fade40fbea0 msgr2=0x7fade40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 -- 172.21.15.32:0/229972227 shutdown_connections 2023-08-11T15:45:57.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.850+0000 7fadebbff700 1 -- 172.21.15.32:0/229972227 wait complete. 2023-08-11T15:45:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.851+0000 7fadebbff700 1 Processor -- start 2023-08-11T15:45:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.851+0000 7fadebbff700 1 -- start start 2023-08-11T15:45:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadebbff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4100930 0x7fade4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadebbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4101b20 0x7fade4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadebbff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 0x7fade419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadebbff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41a0380 con 0x7fade41024c0 2023-08-11T15:45:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadebbff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41a04c0 con 0x7fade4100930 2023-08-11T15:45:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadebbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41a0650 con 0x7fade4101b20 2023-08-11T15:45:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadea19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 0x7fade419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fade999b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4100930 0x7fade4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fade919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4101b20 0x7fade4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadea19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 0x7fade419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42014/0 (socket says 172.21.15.32:42014) 2023-08-11T15:45:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fade919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4101b20 0x7fade4198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:45470/0 (socket says 172.21.15.32:45470) 2023-08-11T15:45:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fadea19c700 1 -- 172.21.15.32:0/4002288575 learned_addr learned my addr 172.21.15.32:0/4002288575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fade999b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4100930 0x7fade4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:40174/0 (socket says 172.21.15.32:40174) 2023-08-11T15:45:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fade999b700 1 -- 172.21.15.32:0/4002288575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4101b20 msgr2=0x7fade4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fade999b700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4101b20 0x7fade4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.852+0000 7fade999b700 1 -- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 msgr2=0x7fade419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7fade999b700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 0x7fade419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7fade999b700 1 -- 172.21.15.32:0/4002288575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40095f0 con 0x7fade4100930 2023-08-11T15:45:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7fade919a700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4101b20 0x7fade4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7fadea19c700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 0x7fade419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7fade999b700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4100930 0x7fade4198800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fade000e9c0 tx=0x7fade000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7faddaffd700 1 -- 172.21.15.32:0/4002288575 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0004220 con 0x7fade4100930 2023-08-11T15:45:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade41a0930 con 0x7fade4100930 2023-08-11T15:45:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7faddaffd700 1 -- 172.21.15.32:0/4002288575 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fade0004380 con 0x7fade4100930 2023-08-11T15:45:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7faddaffd700 1 -- 172.21.15.32:0/4002288575 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0003be0 con 0x7fade4100930 2023-08-11T15:45:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.853+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade41a0e90 con 0x7fade4100930 2023-08-11T15:45:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.857+0000 7faddaffd700 1 -- 172.21.15.32:0/4002288575 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fade0003d40 con 0x7fade4100930 2023-08-11T15:45:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 Processor -- start 2023-08-11T15:45:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 -- start start 2023-08-11T15:45:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3768215e0 con 0x55f3768c8a80 2023-08-11T15:45:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37681ef00 con 0x55f3768c9da0 2023-08-11T15:45:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.858+0000 7fadff452b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37681a7f0 con 0x55f3768c93e0 2023-08-11T15:45:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42018/0 (socket says 172.21.15.32:42018) 2023-08-11T15:45:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 -- 172.21.15.32:0/597708282 learned_addr learned my addr 172.21.15.32:0/597708282 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fade999b700 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fade919a700 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7faddaffd700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fadd00a0060 0x7fadd00a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7faddaffd700 1 -- 172.21.15.32:0/4002288575 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fade0014070 con 0x7fade4100930 2023-08-11T15:45:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 -- 172.21.15.32:0/597708282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 msgr2=0x55f3768c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 -- 172.21.15.32:0/597708282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 msgr2=0x55f3768ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 -- 172.21.15.32:0/597708282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f37669a5e0 con 0x55f3768c8a80 2023-08-11T15:45:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fade919a700 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fadd00a0060 0x7fadd00a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.859+0000 7fadea19c700 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768c8ea0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faddc007920 tx=0x7faddc007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3290dfe8c075ab85 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.860+0000 7fadcffff700 1 -- 172.21.15.32:0/597708282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc010ea0 con 0x55f3768c8a80 2023-08-11T15:45:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.860+0000 7fadcffff700 1 -- 172.21.15.32:0/597708282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc026070 con 0x55f3768c8a80 2023-08-11T15:45:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.860+0000 7fadcffff700 1 -- 172.21.15.32:0/597708282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc017b90 con 0x55f3768c8a80 2023-08-11T15:45:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.860+0000 7fadea19c700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fadd00a0060 0x7fadd00a2520 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faddc021000 tx=0x7faddc01e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 -- 172.21.15.32:0/597708282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 msgr2=0x55f3768c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768c8ea0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faddc007920 tx=0x7faddc007c30 comp rx=0 tx=0).stop 2023-08-11T15:45:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 -- 172.21.15.32:0/597708282 shutdown_connections 2023-08-11T15:45:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768c8ea0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 --2- 172.21.15.32:0/597708282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 -- 172.21.15.32:0/597708282 >> 172.21.15.32:0/597708282 conn(0x55f3767f5900 msgr2=0x55f3768c33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 -- 172.21.15.32:0/597708282 shutdown_connections 2023-08-11T15:45:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 -- 172.21.15.32:0/597708282 wait complete. 2023-08-11T15:45:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.861+0000 7fadff452b80 1 -- start start 2023-08-11T15:45:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 1 Processor -- start 2023-08-11T15:45:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c8a80 0x55f3768d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c93e0 0x55f3768d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3767b8a70 con 0x55f3768c93e0 2023-08-11T15:45:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3767b8770 con 0x55f3768c9da0 2023-08-11T15:45:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37669c8d0 con 0x55f3768c8a80 2023-08-11T15:45:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadff452b80 10 client.0 _ll_register_callbacks cb 0x55f3767bba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:45:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fade919a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c93e0 0x55f3768d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fade999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c8a80 0x55f3768d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fade919a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c93e0 0x55f3768d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42024/0 (socket says 172.21.15.32:42024) 2023-08-11T15:45:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fade919a700 1 -- 172.21.15.32:0/3976799171 learned_addr learned my addr 172.21.15.32:0/3976799171 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.862+0000 7fadea19c700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fade999b700 1 -- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 msgr2=0x55f3768d9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fade999b700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fade999b700 1 -- 172.21.15.32:0/3976799171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c93e0 msgr2=0x55f3768d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fade999b700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c93e0 0x55f3768d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fade999b700 1 -- 172.21.15.32:0/3976799171 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade41096c0 con 0x55f3768c8a80 2023-08-11T15:45:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fadea19c700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fade919a700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c93e0 0x55f3768d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:45:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fade999b700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c8a80 0x55f3768d20a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fade00c8fd0 tx=0x7fade0098d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e762680a3e5f86 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fadcd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:45:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fadcd7fa700 1 -- 172.21.15.32:0/3976799171 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0099370 con 0x55f3768c8a80 2023-08-11T15:45:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3768bba40 con 0x55f3768c8a80 2023-08-11T15:45:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fadcd7fa700 1 -- 172.21.15.32:0/3976799171 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fade00044f0 con 0x55f3768c8a80 2023-08-11T15:45:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fadcd7fa700 1 -- 172.21.15.32:0/3976799171 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade0099610 con 0x55f3768c8a80 2023-08-11T15:45:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.863+0000 7fadff452b80 10 client.25153 fetch_fsmap learned FSMap version 21 2023-08-11T15:45:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7faddc007500 con 0x55f3768c8a80 2023-08-11T15:45:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadcd7fa700 1 -- 172.21.15.32:0/3976799171 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3768bba40 con 0x55f3768c8a80 2023-08-11T15:45:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadcd7fa700 1 -- 172.21.15.32:0/3976799171 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fade00a0890 con 0x55f3768c8a80 2023-08-11T15:45:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadff452b80 10 client.25153 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:45:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadff452b80 10 client.25153 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:45:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadff452b80 20 client.25153 populate_metadata read hostname 'smithi032' 2023-08-11T15:45:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadff452b80 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3768ddc20 0x55f3768dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadff452b80 4 client.25153 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:45:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.864+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f376897490 con 0x55f3768ddc20 2023-08-11T15:45:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.865+0000 7fade919a700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3768ddc20 0x55f3768dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.865+0000 7fade919a700 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3768ddc20 0x55f3768dc780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3486b49e1fa802 server_cookie=c69a17ae42ac3e80 in_seq=0 out_seq=0 2023-08-11T15:45:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.865+0000 7fadcd7fa700 10 client.25153 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:45:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadcd7fa700 1 -- 172.21.15.32:0/3976799171 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f376897490 con 0x55f3768ddc20 2023-08-11T15:45:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadcd7fa700 10 client.25153 handle_command_reply: tid=0 2023-08-11T15:45:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 client.25153 shutdown 2023-08-11T15:45:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c8a80 msgr2=0x55f3768d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c8a80 0x55f3768d20a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fade00c8fd0 tx=0x7fade0098d60 comp rx=0 tx=0).stop 2023-08-11T15:45:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 shutdown_connections 2023-08-11T15:45:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3768ddc20 0x55f3768dc780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadcd7fa700 0 client.25153 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:45:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c8a80 0x55f3768d20a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 --2- 172.21.15.32:0/3976799171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c93e0 0x55f3768d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 >> 172.21.15.32:0/3976799171 conn(0x55f3767f5900 msgr2=0x55f3768bdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 shutdown_connections 2023-08-11T15:45:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.866+0000 7fadff452b80 1 -- 172.21.15.32:0/3976799171 wait complete. 2023-08-11T15:45:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.867+0000 7fadff452b80 20 client.25153 trim_cache size 0 max 16384 2023-08-11T15:45:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 Processor -- start 2023-08-11T15:45:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 -- start start 2023-08-11T15:45:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c8a80 0x55f3768ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c9da0 0x55f3768d95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3767e3ef0 con 0x55f3768c9da0 2023-08-11T15:45:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3767d7480 con 0x55f3768c8a80 2023-08-11T15:45:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadff452b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3767e4c30 con 0x55f3768c93e0 2023-08-11T15:45:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.885+0000 7fadea19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c9da0 0x55f3768d95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c9da0 0x55f3768d95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42036/0 (socket says 172.21.15.32:42036) 2023-08-11T15:45:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 -- 172.21.15.32:0/1806317589 learned_addr learned my addr 172.21.15.32:0/1806317589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:57.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fade999b700 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c8a80 0x55f3768ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fade919a700 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 -- 172.21.15.32:0/1806317589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 msgr2=0x55f3768cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 -- 172.21.15.32:0/1806317589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c8a80 msgr2=0x55f3768ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c8a80 0x55f3768ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 -- 172.21.15.32:0/1806317589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f376646d90 con 0x55f3768c9da0 2023-08-11T15:45:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadea19c700 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c9da0 0x55f3768d95e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faddc009050 tx=0x7faddc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7df86146851835 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadd8bff700 1 -- 172.21.15.32:0/1806317589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc017a60 con 0x55f3768c9da0 2023-08-11T15:45:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadd8bff700 1 -- 172.21.15.32:0/1806317589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc00f040 con 0x55f3768c9da0 2023-08-11T15:45:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.886+0000 7fadd8bff700 1 -- 172.21.15.32:0/1806317589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc022c70 con 0x55f3768c9da0 2023-08-11T15:45:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 -- 172.21.15.32:0/1806317589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c9da0 msgr2=0x55f3768d95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c9da0 0x55f3768d95e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faddc009050 tx=0x7faddc005bc0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 -- 172.21.15.32:0/1806317589 shutdown_connections 2023-08-11T15:45:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c9da0 0x55f3768d95e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c8a80 0x55f3768ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 --2- 172.21.15.32:0/1806317589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 -- 172.21.15.32:0/1806317589 >> 172.21.15.32:0/1806317589 conn(0x55f3767f5900 msgr2=0x55f3768d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 -- 172.21.15.32:0/1806317589 shutdown_connections 2023-08-11T15:45:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.887+0000 7fadff452b80 1 -- 172.21.15.32:0/1806317589 wait complete. 2023-08-11T15:45:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 -- start start 2023-08-11T15:45:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 Processor -- start 2023-08-11T15:45:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3768d4b40 con 0x55f3768c8a80 2023-08-11T15:45:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f376931490 con 0x55f3768c9da0 2023-08-11T15:45:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f376931620 con 0x55f3768c93e0 2023-08-11T15:45:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fadff452b80 10 client.0 _ll_register_callbacks cb 0x7fade0094be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:45:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.888+0000 7fade999b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42048/0 (socket says 172.21.15.32:42048) 2023-08-11T15:45:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fadea19c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 -- 172.21.15.32:0/96366185 learned_addr learned my addr 172.21.15.32:0/96366185 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:45:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade919a700 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 -- 172.21.15.32:0/96366185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 msgr2=0x55f3768d9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768d9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 -- 172.21.15.32:0/96366185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 msgr2=0x55f3768d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:45:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 -- 172.21.15.32:0/96366185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f37669a5e0 con 0x55f3768c8a80 2023-08-11T15:45:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fade999b700 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768d3f20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fade0098c10 tx=0x7fade00a0ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588367ea542a6aad server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:45:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fadcd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:45:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fadcd7fa700 1 -- 172.21.15.32:0/96366185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0098400 con 0x55f3768c8a80 2023-08-11T15:45:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faddc026070 con 0x55f3768c8a80 2023-08-11T15:45:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.889+0000 7fadcd7fa700 1 -- 172.21.15.32:0/96366185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fade00044f0 con 0x55f3768c8a80 2023-08-11T15:45:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadcd7fa700 1 -- 172.21.15.32:0/96366185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade0099610 con 0x55f3768c8a80 2023-08-11T15:45:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 10 client.15147 fetch_fsmap learned FSMap version 21 2023-08-11T15:45:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f37662fc10 con 0x55f3768c8a80 2023-08-11T15:45:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadcd7fa700 1 -- 172.21.15.32:0/96366185 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc026070 con 0x55f3768c8a80 2023-08-11T15:45:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadcd7fa700 1 -- 172.21.15.32:0/96366185 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fade00b2030 con 0x55f3768c8a80 2023-08-11T15:45:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 10 client.15147 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:45:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:45:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 20 client.15147 populate_metadata read hostname 'smithi032' 2023-08-11T15:45:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3769363b0 0x55f376932870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:45:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 4 client.15147 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-08-11T15:45:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f3768137d0 con 0x55f3769363b0 2023-08-11T15:45:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.890+0000 7fade919a700 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3769363b0 0x55f376932870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:45:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fade919a700 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3769363b0 0x55f376932870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28ce196942cf386 server_cookie=fd0021c3b354e44f in_seq=0 out_seq=0 2023-08-11T15:45:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadcd7fa700 10 client.15147 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:45:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadcd7fa700 1 -- 172.21.15.32:0/96366185 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f3768137d0 con 0x55f3769363b0 2023-08-11T15:45:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadcd7fa700 10 client.15147 handle_command_reply: tid=0 2023-08-11T15:45:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 client.15147 shutdown 2023-08-11T15:45:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 msgr2=0x55f3768d3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768d3f20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fade0098c10 tx=0x7fade00a0ac0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 shutdown_connections 2023-08-11T15:45:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f3769363b0 0x55f376932870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3768c8a80 0x55f3768d3f20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3768c9da0 0x55f3768d4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadcd7fa700 0 client.15147 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:45:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 --2- 172.21.15.32:0/96366185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f3768c93e0 0x55f3768d9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.891+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 >> 172.21.15.32:0/96366185 conn(0x55f3767f5900 msgr2=0x55f3768ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 shutdown_connections 2023-08-11T15:45:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadff452b80 1 -- 172.21.15.32:0/96366185 wait complete. 2023-08-11T15:45:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadff452b80 20 client.15147 trim_cache size 0 max 16384 2023-08-11T15:45:57.905 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:45:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "cd36a838-385c-11ee-9b37-001a4aab830c", 2023-08-11T15:45:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-08-11T15:45:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24578, 2023-08-11T15:45:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-08-11T15:45:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-08-11T15:45:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-08-11T15:45:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 97.728130763999999, 2023-08-11T15:45:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 21, 2023-08-11T15:45:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 92, 2023-08-11T15:45:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 88, 2023-08-11T15:45:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 105.92751699199999 2023-08-11T15:45:57.907 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:45:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fadd00a0060 msgr2=0x7fadd00a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fadd00a0060 0x7fadd00a2520 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faddc021000 tx=0x7faddc01e000 comp rx=0 tx=0).stop 2023-08-11T15:45:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4100930 msgr2=0x7fade4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:45:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4100930 0x7fade4198800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fade000e9c0 tx=0x7fade000ecd0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 shutdown_connections 2023-08-11T15:45:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fadd00a0060 0x7fadd00a2520 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade41024c0 0x7fade419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fade4100930 0x7fade4198800 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 --2- 172.21.15.32:0/4002288575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade4101b20 0x7fade4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:45:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.892+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 >> 172.21.15.32:0/4002288575 conn(0x7fade40fbea0 msgr2=0x7fade4107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:45:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.893+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 shutdown_connections 2023-08-11T15:45:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:45:57.893+0000 7fadebbff700 1 -- 172.21.15.32:0/4002288575 wait complete. 2023-08-11T15:45:57.912 INFO:tasks.cephadm:Setup complete, yielding 2023-08-11T15:45:57.913 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-11T15:45:57.923 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:45:57.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- bash -c 'ceph orch status' 2023-08-11T15:45:58.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:58 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1835667322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:58.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:58 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1835667322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:58.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:58 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1835667322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:45:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:45:59.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:45:59 smithi134 ceph-mon[130447]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:45:59.550 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:45:59 smithi032 ceph-mon[97103]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:45:59.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:45:59 smithi192 ceph-mon[130806]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 -- 172.21.15.32:0/3750127484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144101bb0 msgr2=0x7f9144102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:00.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3750127484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144101bb0 0x7f9144102010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91340099c0 tx=0x7f9134009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 -- 172.21.15.32:0/3750127484 shutdown_connections 2023-08-11T15:46:00.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3750127484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9144102550 0x7f9144108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3750127484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91441009b0 0x7f9144100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3750127484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144101bb0 0x7f9144102010 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 -- 172.21.15.32:0/3750127484 >> 172.21.15.32:0/3750127484 conn(0x7f91440fbf10 msgr2=0x7f91440fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:00.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 -- 172.21.15.32:0/3750127484 shutdown_connections 2023-08-11T15:46:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.170+0000 7f914b9f1700 1 -- 172.21.15.32:0/3750127484 wait complete. 2023-08-11T15:46:00.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 Processor -- start 2023-08-11T15:46:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 -- start start 2023-08-11T15:46:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91441009b0 0x7f91441966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9144101bb0 0x7f9144196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144102550 0x7f914419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914419e250 con 0x7f9144102550 2023-08-11T15:46:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914419e390 con 0x7f9144101bb0 2023-08-11T15:46:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.171+0000 7f914b9f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914419e520 con 0x7f91441009b0 2023-08-11T15:46:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144102550 0x7f914419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144102550 0x7f914419dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42076/0 (socket says 172.21.15.32:42076) 2023-08-11T15:46:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 -- 172.21.15.32:0/3812699888 learned_addr learned my addr 172.21.15.32:0/3812699888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f914978d700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91441009b0 0x7f91441966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9148f8c700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9144101bb0 0x7f9144196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 -- 172.21.15.32:0/3812699888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91441009b0 msgr2=0x7f91441966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91441009b0 0x7f91441966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 -- 172.21.15.32:0/3812699888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9144101bb0 msgr2=0x7f9144196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9144101bb0 0x7f9144196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 -- 172.21.15.32:0/3812699888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340095f0 con 0x7f9144102550 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.172+0000 7f9149f8e700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144102550 0x7f914419dc80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f913c007f10 tx=0x7f913c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.173+0000 7f913a7fc700 1 -- 172.21.15.32:0/3812699888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00dd20 con 0x7f9144102550 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.173+0000 7f913a7fc700 1 -- 172.21.15.32:0/3812699888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913c00f040 con 0x7f9144102550 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.173+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914419e800 con 0x7f9144102550 2023-08-11T15:46:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.173+0000 7f913a7fc700 1 -- 172.21.15.32:0/3812699888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0048a0 con 0x7f9144102550 2023-08-11T15:46:00.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.173+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914419ecb0 con 0x7f9144102550 2023-08-11T15:46:00.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.174+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914410c200 con 0x7f9144102550 2023-08-11T15:46:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.176+0000 7f913a7fc700 1 -- 172.21.15.32:0/3812699888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f913c003a00 con 0x7f9144102550 2023-08-11T15:46:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.177+0000 7f913a7fc700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f913009ffe0 0x7f91300a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.177+0000 7f913a7fc700 1 -- 172.21.15.32:0/3812699888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f913c0caf60 con 0x7f9144102550 2023-08-11T15:46:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.177+0000 7f914978d700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f913009ffe0 0x7f91300a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.178+0000 7f914978d700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f913009ffe0 0x7f91300a24a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9144101a10 tx=0x7f914000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.181+0000 7f913a7fc700 1 -- 172.21.15.32:0/3812699888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f913c04ec30 con 0x7f9144102550 2023-08-11T15:46:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.410+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9144060d60 con 0x7f913009ffe0 2023-08-11T15:46:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.411+0000 7f913a7fc700 1 -- 172.21.15.32:0/3812699888 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9144060d60 con 0x7f913009ffe0 2023-08-11T15:46:00.413 INFO:teuthology.orchestra.run.smithi032.stdout:Backend: cephadm 2023-08-11T15:46:00.413 INFO:teuthology.orchestra.run.smithi032.stdout:Available: Yes 2023-08-11T15:46:00.413 INFO:teuthology.orchestra.run.smithi032.stdout:Paused: No 2023-08-11T15:46:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f913009ffe0 msgr2=0x7f91300a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f913009ffe0 0x7f91300a24a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9144101a10 tx=0x7f914000b410 comp rx=0 tx=0).stop 2023-08-11T15:46:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144102550 msgr2=0x7f914419dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144102550 0x7f914419dc80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f913c007f10 tx=0x7f913c00d3b0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 shutdown_connections 2023-08-11T15:46:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f913009ffe0 0x7f91300a24a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9144102550 0x7f914419dc80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91441009b0 0x7f91441966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 --2- 172.21.15.32:0/3812699888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9144101bb0 0x7f9144196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 >> 172.21.15.32:0/3812699888 conn(0x7f91440fbf10 msgr2=0x7f91440fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 shutdown_connections 2023-08-11T15:46:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:00.414+0000 7f914b9f1700 1 -- 172.21.15.32:0/3812699888 wait complete. 2023-08-11T15:46:00.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- bash -c 'ceph orch ps' 2023-08-11T15:46:01.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:01 smithi134 ceph-mon[130447]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:01.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:01 smithi032 ceph-mon[97103]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:01.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:01 smithi192 ceph-mon[130806]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:01.693 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:46:02.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:02 smithi134 ceph-mon[130447]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:02 smithi032 ceph-mon[97103]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:02 smithi192 ceph-mon[130806]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 -- 172.21.15.32:0/725673906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 msgr2=0x7fbdc8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 --2- 172.21.15.32:0/725673906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 0x7fbdc8102010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009920 tx=0x7fbdbc009c30 comp rx=0 tx=0).stop 2023-08-11T15:46:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 -- 172.21.15.32:0/725673906 shutdown_connections 2023-08-11T15:46:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 --2- 172.21.15.32:0/725673906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8102550 0x7fbdc8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 --2- 172.21.15.32:0/725673906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc81009b0 0x7fbdc8100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 --2- 172.21.15.32:0/725673906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 0x7fbdc8102010 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 -- 172.21.15.32:0/725673906 >> 172.21.15.32:0/725673906 conn(0x7fbdc80fbf10 msgr2=0x7fbdc80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 -- 172.21.15.32:0/725673906 shutdown_connections 2023-08-11T15:46:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.196+0000 7fbdccced700 1 -- 172.21.15.32:0/725673906 wait complete. 2023-08-11T15:46:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.197+0000 7fbdccced700 1 Processor -- start 2023-08-11T15:46:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.197+0000 7fbdccced700 1 -- start start 2023-08-11T15:46:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdccced700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc81009b0 0x7fbdc80746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdccced700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 0x7fbdc8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdccced700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8102550 0x7fbdc8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdccced700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80739d0 con 0x7fbdc8101bb0 2023-08-11T15:46:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdccced700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc8073b40 con 0x7fbdc8102550 2023-08-11T15:46:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdccced700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81a0770 con 0x7fbdc81009b0 2023-08-11T15:46:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc81009b0 0x7fbdc80746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 0x7fbdc8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc81009b0 0x7fbdc80746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:45544/0 (socket says 172.21.15.32:45544) 2023-08-11T15:46:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc659c700 1 -- 172.21.15.32:0/3760695613 learned_addr learned my addr 172.21.15.32:0/3760695613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc6d9d700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8102550 0x7fbdc8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc5d9b700 1 -- 172.21.15.32:0/3760695613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc81009b0 msgr2=0x7fbdc80746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc5d9b700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc81009b0 0x7fbdc80746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc5d9b700 1 -- 172.21.15.32:0/3760695613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8102550 msgr2=0x7fbdc8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.198+0000 7fbdc5d9b700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8102550 0x7fbdc8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.199+0000 7fbdc5d9b700 1 -- 172.21.15.32:0/3760695613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc0009d00 con 0x7fbdc8101bb0 2023-08-11T15:46:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.199+0000 7fbdc5d9b700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 0x7fbdc8072d10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc000c00 tx=0x7fbdbc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.199+0000 7fbdb77fe700 1 -- 172.21.15.32:0/3760695613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc01c070 con 0x7fbdc8101bb0 2023-08-11T15:46:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.199+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc009520 con 0x7fbdc8101bb0 2023-08-11T15:46:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.199+0000 7fbdb77fe700 1 -- 172.21.15.32:0/3760695613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdbc00f7d0 con 0x7fbdc8101bb0 2023-08-11T15:46:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.199+0000 7fbdb77fe700 1 -- 172.21.15.32:0/3760695613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0038e0 con 0x7fbdc8101bb0 2023-08-11T15:46:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.199+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc81a0c70 con 0x7fbdc8101bb0 2023-08-11T15:46:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.200+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc8108080 con 0x7fbdc8101bb0 2023-08-11T15:46:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.202+0000 7fbdb77fe700 1 -- 172.21.15.32:0/3760695613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fbdbc020030 con 0x7fbdc8101bb0 2023-08-11T15:46:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.204+0000 7fbdb77fe700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbdb009ffe0 0x7fbdb00a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.204+0000 7fbdb77fe700 1 -- 172.21.15.32:0/3760695613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbdbc0cc680 con 0x7fbdc8101bb0 2023-08-11T15:46:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.204+0000 7fbdc659c700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbdb009ffe0 0x7fbdb00a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.205+0000 7fbdc659c700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbdb009ffe0 0x7fbdb00a24a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800bd70 tx=0x7fbdb800b4a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.208+0000 7fbdb77fe700 1 -- 172.21.15.32:0/3760695613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbdbc095fb0 con 0x7fbdc8101bb0 2023-08-11T15:46:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.431+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdc802ce00 con 0x7fbdb009ffe0 2023-08-11T15:46:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.444+0000 7fbdb77fe700 1 -- 172.21.15.32:0/3760695613 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fbdc802ce00 con 0x7fbdb009ffe0 2023-08-11T15:46:03.446 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-08-11T15:46:03.446 INFO:teuthology.orchestra.run.smithi032.stdout:mds.a smithi032 running (114s) - 113s 17.3M - 18.0.0-5437-g92684807 a2667c7aa6ee 016a7e821f64 2023-08-11T15:46:03.446 INFO:teuthology.orchestra.run.smithi032.stdout:mds.b smithi134 running (111s) - 111s 29.4M - 18.0.0-5437-g92684807 a2667c7aa6ee 02f482ab81ac 2023-08-11T15:46:03.447 INFO:teuthology.orchestra.run.smithi032.stdout:mds.c smithi192 running (2m) - 2m 16.4M - 18.0.0-5437-g92684807 a2667c7aa6ee 47959591fee3 2023-08-11T15:46:03.447 INFO:teuthology.orchestra.run.smithi032.stdout:mds.d smithi032 running (2m) - 2m 13.8M - 18.0.0-5437-g92684807 a2667c7aa6ee 3c3d60154cd9 2023-08-11T15:46:03.447 INFO:teuthology.orchestra.run.smithi032.stdout:mds.e smithi134 running (2m) - 2m 15.1M - 18.0.0-5437-g92684807 a2667c7aa6ee f7dce63560e9 2023-08-11T15:46:03.448 INFO:teuthology.orchestra.run.smithi032.stdout:mds.f smithi192 running (2m) - 2m 15.1M - 18.0.0-5437-g92684807 a2667c7aa6ee 68c0317b4379 2023-08-11T15:46:03.448 INFO:teuthology.orchestra.run.smithi032.stdout:mds.g smithi032 running (117s) - 116s 13.7M - 18.0.0-5437-g92684807 a2667c7aa6ee 85b2f72b4dfe 2023-08-11T15:46:03.448 INFO:teuthology.orchestra.run.smithi032.stdout:mds.h smithi134 running (2m) - 2m 17.3M - 18.0.0-5437-g92684807 a2667c7aa6ee c7bd13c5307e 2023-08-11T15:46:03.448 INFO:teuthology.orchestra.run.smithi032.stdout:mds.i smithi192 running (2m) - 119s 16.5M - 18.0.0-5437-g92684807 a2667c7aa6ee 9372634112bf 2023-08-11T15:46:03.448 INFO:teuthology.orchestra.run.smithi032.stdout:mds.j smithi032 running (2m) - 2m 13.8M - 18.0.0-5437-g92684807 a2667c7aa6ee 10ecbc4db939 2023-08-11T15:46:03.449 INFO:teuthology.orchestra.run.smithi032.stdout:mds.k smithi134 running (2m) - 2m 17.5M - 18.0.0-5437-g92684807 a2667c7aa6ee 3d38f3278566 2023-08-11T15:46:03.449 INFO:teuthology.orchestra.run.smithi032.stdout:mds.l smithi192 running (2m) - 2m 15.0M - 18.0.0-5437-g92684807 a2667c7aa6ee f9d10d5e8d4b 2023-08-11T15:46:03.449 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.x smithi032 *:9283,8765 running (8m) - 8m 526M - 18.0.0-5437-g92684807 a2667c7aa6ee 140588cec60c 2023-08-11T15:46:03.449 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.y smithi134 *:8443,8765 running (7m) - 7m 449M - 18.0.0-5437-g92684807 a2667c7aa6ee 2387ae3fb3e6 2023-08-11T15:46:03.449 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.z smithi192 *:8443,8765 running (7m) - 7m 450M - 18.0.0-5437-g92684807 a2667c7aa6ee 61c16becd479 2023-08-11T15:46:03.449 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a smithi032 running (9m) - 9m 94.1M 2048M 18.0.0-5437-g92684807 a2667c7aa6ee 206d2b89c8f6 2023-08-11T15:46:03.449 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b smithi134 running (7m) - 7m 64.7M 2048M 18.0.0-5437-g92684807 a2667c7aa6ee 266dc0a763d5 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c smithi192 running (7m) - 7m 63.3M 2048M 18.0.0-5437-g92684807 a2667c7aa6ee f9c31a36d07c 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (6m) - 6m 69.9M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee 5d5310c6bea0 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi134 running (6m) - 6m 78.5M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee e1d296d8429b 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi192 running (6m) - 6m 77.3M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 2873af3be1ad 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi032 running (5m) - 5m 65.5M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee 4662f03b93ad 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi134 running (5m) - 5m 62.9M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 2b3d49ea4367 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi192 running (4m) - 4m 70.2M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 60efa49dfcc9 2023-08-11T15:46:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 smithi032 running (4m) - 4m 60.7M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee 924332c6fa76 2023-08-11T15:46:03.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 smithi134 running (4m) - 4m 63.8M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 4a3b5db87192 2023-08-11T15:46:03.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 smithi192 running (3m) - 3m 62.5M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 1da80a8b1d13 2023-08-11T15:46:03.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 smithi032 running (3m) - 3m 59.2M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee d5a4a67e7b12 2023-08-11T15:46:03.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 smithi134 running (2m) - 2m 64.3M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 22e240b32932 2023-08-11T15:46:03.451 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 smithi192 running (2m) - 2m 55.3M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee e082cfabbd73 2023-08-11T15:46:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbdb009ffe0 msgr2=0x7fbdb00a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbdb009ffe0 0x7fbdb00a24a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800bd70 tx=0x7fbdb800b4a0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 msgr2=0x7fbdc8072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 0x7fbdc8072d10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc000c00 tx=0x7fbdbc00b790 comp rx=0 tx=0).stop 2023-08-11T15:46:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 shutdown_connections 2023-08-11T15:46:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbdb009ffe0 0x7fbdb00a24a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc8102550 0x7fbdc8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc81009b0 0x7fbdc80746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 --2- 172.21.15.32:0/3760695613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc8101bb0 0x7fbdc8072d10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 >> 172.21.15.32:0/3760695613 conn(0x7fbdc80fbf10 msgr2=0x7fbdc81051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.446+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 shutdown_connections 2023-08-11T15:46:03.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:03.447+0000 7fbdccced700 1 -- 172.21.15.32:0/3760695613 wait complete. 2023-08-11T15:46:03.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:03 smithi134 ceph-mon[130447]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:03.471 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:03 smithi032 ceph-mon[97103]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:03.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:03 smithi192 ceph-mon[130806]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:03.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- bash -c 'ceph orch ls' 2023-08-11T15:46:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:46:05.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:05 smithi032 ceph-mon[97103]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:05.264 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:05 smithi032 ceph-mon[97103]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:05.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:05 smithi134 ceph-mon[130447]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:05.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:05 smithi134 ceph-mon[130447]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:05.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:05 smithi192 ceph-mon[130806]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:05.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:05 smithi192 ceph-mon[130806]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T15:46:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.151+0000 7f00276ad700 1 -- 172.21.15.32:0/808444611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020108880 msgr2=0x7f002010ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.151+0000 7f00276ad700 1 --2- 172.21.15.32:0/808444611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020108880 0x7f002010ad10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f001800b0a0 tx=0x7f001800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.151+0000 7f00276ad700 1 -- 172.21.15.32:0/808444611 shutdown_connections 2023-08-11T15:46:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.151+0000 7f00276ad700 1 --2- 172.21.15.32:0/808444611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020108880 0x7f002010ad10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.151+0000 7f00276ad700 1 --2- 172.21.15.32:0/808444611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0020074950 0x7f0020072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.151+0000 7f00276ad700 1 --2- 172.21.15.32:0/808444611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0020073380 0x7f0020073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.151+0000 7f00276ad700 1 -- 172.21.15.32:0/808444611 >> 172.21.15.32:0/808444611 conn(0x7f00200fbf10 msgr2=0x7f00200fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.152+0000 7f00276ad700 1 -- 172.21.15.32:0/808444611 shutdown_connections 2023-08-11T15:46:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.152+0000 7f00276ad700 1 -- 172.21.15.32:0/808444611 wait complete. 2023-08-11T15:46:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 Processor -- start 2023-08-11T15:46:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 -- start start 2023-08-11T15:46:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020073380 0x7f0020198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0020074950 0x7f0020198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0020108880 0x7f002019ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201a0510 con 0x7f0020073380 2023-08-11T15:46:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201a0650 con 0x7f0020108880 2023-08-11T15:46:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f00276ad700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201a07e0 con 0x7f0020074950 2023-08-11T15:46:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.153+0000 7f0025449700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020073380 0x7f0020198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020073380 0x7f0020198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57692/0 (socket says 172.21.15.32:57692) 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 -- 172.21.15.32:0/1035636988 learned_addr learned my addr 172.21.15.32:0/1035636988 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0024c48700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0020074950 0x7f0020198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025c4a700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0020108880 0x7f002019ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 -- 172.21.15.32:0/1035636988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0020074950 msgr2=0x7f0020198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0020074950 0x7f0020198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 -- 172.21.15.32:0/1035636988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0020108880 msgr2=0x7f002019ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0020108880 0x7f002019ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 -- 172.21.15.32:0/1035636988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0018009d00 con 0x7f0020073380 2023-08-11T15:46:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.154+0000 7f0025449700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020073380 0x7f0020198980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d9b0 tx=0x7f001c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.155+0000 7f00167fc700 1 -- 172.21.15.32:0/1035636988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c009930 con 0x7f0020073380 2023-08-11T15:46:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.155+0000 7f00167fc700 1 -- 172.21.15.32:0/1035636988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001c010460 con 0x7f0020073380 2023-08-11T15:46:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.155+0000 7f00167fc700 1 -- 172.21.15.32:0/1035636988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00f620 con 0x7f0020073380 2023-08-11T15:46:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.155+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00201a0ac0 con 0x7f0020073380 2023-08-11T15:46:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.155+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00201a0fc0 con 0x7f0020073380 2023-08-11T15:46:06.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.156+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0020106f30 con 0x7f0020073380 2023-08-11T15:46:06.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.158+0000 7f00167fc700 1 -- 172.21.15.32:0/1035636988 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f001c0108d0 con 0x7f0020073380 2023-08-11T15:46:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.159+0000 7f00167fc700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f000c0a0060 0x7f000c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.160+0000 7f00167fc700 1 -- 172.21.15.32:0/1035636988 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f001c0cb0d0 con 0x7f0020073380 2023-08-11T15:46:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.160+0000 7f0024c48700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f000c0a0060 0x7f000c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.160+0000 7f0024c48700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f000c0a0060 0x7f000c0a2520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0020074680 tx=0x7f001000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.161+0000 7f00167fc700 1 -- 172.21.15.32:0/1035636988 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f001c0949f0 con 0x7f0020073380 2023-08-11T15:46:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.378+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f002002cee0 con 0x7f000c0a0060 2023-08-11T15:46:06.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.384+0000 7f00167fc700 1 -- 172.21.15.32:0/1035636988 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f002002cee0 con 0x7f000c0a0060 2023-08-11T15:46:06.385 INFO:teuthology.orchestra.run.smithi032.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-08-11T15:46:06.386 INFO:teuthology.orchestra.run.smithi032.stdout:mds.all 12/12 - 2m smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-08-11T15:46:06.386 INFO:teuthology.orchestra.run.smithi032.stdout:mgr 3/3 - 7m smithi032=x;smithi134=y;smithi192=z;count:3 2023-08-11T15:46:06.386 INFO:teuthology.orchestra.run.smithi032.stdout:mon 3/3 - 7m smithi032:172.21.15.32=a;smithi134:172.21.15.134=b;smithi192:172.21.15.192=c;count:3 2023-08-11T15:46:06.387 INFO:teuthology.orchestra.run.smithi032.stdout:osd 12 - - 2023-08-11T15:46:06.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f000c0a0060 msgr2=0x7f000c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:06.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f000c0a0060 0x7f000c0a2520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0020074680 tx=0x7f001000b410 comp rx=0 tx=0).stop 2023-08-11T15:46:06.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020073380 msgr2=0x7f0020198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020073380 0x7f0020198980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d9b0 tx=0x7f001c00dcc0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 shutdown_connections 2023-08-11T15:46:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f000c0a0060 0x7f000c0a2520 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0020073380 0x7f0020198980 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0020108880 0x7f002019ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 --2- 172.21.15.32:0/1035636988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0020074950 0x7f0020198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 >> 172.21.15.32:0/1035636988 conn(0x7f00200fbf10 msgr2=0x7f00200fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.386+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 shutdown_connections 2023-08-11T15:46:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:06.387+0000 7f00276ad700 1 -- 172.21.15.32:0/1035636988 wait complete. 2023-08-11T15:46:06.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- bash -c 'ceph orch host ls' 2023-08-11T15:46:07.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:07 smithi134 ceph-mon[130447]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T15:46:07.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:07 smithi032 ceph-mon[97103]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T15:46:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:46:07.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:07 smithi192 ceph-mon[130806]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T15:46:08.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:08 smithi032 ceph-mon[97103]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:08.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:08 smithi192 ceph-mon[130806]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:08.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:08 smithi134 ceph-mon[130447]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:09.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 -- 172.21.15.32:0/102810691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214101bb0 msgr2=0x7f8214102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 --2- 172.21.15.32:0/102810691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214101bb0 0x7f8214102010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f82080099c0 tx=0x7f8208009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 -- 172.21.15.32:0/102810691 shutdown_connections 2023-08-11T15:46:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 --2- 172.21.15.32:0/102810691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8214102550 0x7f8214108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 --2- 172.21.15.32:0/102810691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82141009b0 0x7f8214100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 --2- 172.21.15.32:0/102810691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214101bb0 0x7f8214102010 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 -- 172.21.15.32:0/102810691 >> 172.21.15.32:0/102810691 conn(0x7f82140fbf10 msgr2=0x7f82140fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 -- 172.21.15.32:0/102810691 shutdown_connections 2023-08-11T15:46:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.104+0000 7f821a7cf700 1 -- 172.21.15.32:0/102810691 wait complete. 2023-08-11T15:46:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.105+0000 7f821a7cf700 1 Processor -- start 2023-08-11T15:46:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.105+0000 7f821a7cf700 1 -- start start 2023-08-11T15:46:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f821a7cf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82141009b0 0x7f82140746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f821a7cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214101bb0 0x7f8214072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f821a7cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214102550 0x7f8214073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:09.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f821a7cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82140739d0 con 0x7f8214102550 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f821a7cf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214073b40 con 0x7f82141009b0 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f821a7cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a0730 con 0x7f8214101bb0 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f82137fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214101bb0 0x7f8214072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8213fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82141009b0 0x7f82140746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8218d6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214102550 0x7f8214073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f82137fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214101bb0 0x7f8214072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:55742/0 (socket says 172.21.15.32:55742) 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f82137fe700 1 -- 172.21.15.32:0/1920993030 learned_addr learned my addr 172.21.15.32:0/1920993030 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8213fff700 1 -- 172.21.15.32:0/1920993030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214101bb0 msgr2=0x7f8214072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8213fff700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214101bb0 0x7f8214072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8213fff700 1 -- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214102550 msgr2=0x7f8214073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8213fff700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214102550 0x7f8214073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8213fff700 1 -- 172.21.15.32:0/1920993030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080095f0 con 0x7f82141009b0 2023-08-11T15:46:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f82137fe700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214101bb0 0x7f8214072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.106+0000 7f8218d6c700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214102550 0x7f8214073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.107+0000 7f8213fff700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82141009b0 0x7f82140746c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f820400e9c0 tx=0x7f820400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.107+0000 7f82117fa700 1 -- 172.21.15.32:0/1920993030 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204004220 con 0x7f82141009b0 2023-08-11T15:46:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.107+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82141a0a10 con 0x7f82141009b0 2023-08-11T15:46:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.107+0000 7f82117fa700 1 -- 172.21.15.32:0/1920993030 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8204004380 con 0x7f82141009b0 2023-08-11T15:46:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.107+0000 7f82117fa700 1 -- 172.21.15.32:0/1920993030 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204003be0 con 0x7f82141009b0 2023-08-11T15:46:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.107+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a0f60 con 0x7f82141009b0 2023-08-11T15:46:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.108+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821410c200 con 0x7f82141009b0 2023-08-11T15:46:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.111+0000 7f82117fa700 1 -- 172.21.15.32:0/1920993030 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f8204003d40 con 0x7f82141009b0 2023-08-11T15:46:09.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.112+0000 7f82117fa700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81fc09ffe0 0x7f81fc0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:09.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.112+0000 7f82117fa700 1 -- 172.21.15.32:0/1920993030 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8204014070 con 0x7f82141009b0 2023-08-11T15:46:09.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.112+0000 7f82137fe700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81fc09ffe0 0x7f81fc0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:09.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.113+0000 7f82137fe700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81fc09ffe0 0x7f81fc0a24a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8208004800 tx=0x7f820801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:09.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.114+0000 7f82117fa700 1 -- 172.21.15.32:0/1920993030 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f82040953e0 con 0x7f82141009b0 2023-08-11T15:46:09.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.329+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8214060d60 con 0x7f81fc09ffe0 2023-08-11T15:46:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.331+0000 7f82117fa700 1 -- 172.21.15.32:0/1920993030 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f8214060d60 con 0x7f81fc09ffe0 2023-08-11T15:46:09.332 INFO:teuthology.orchestra.run.smithi032.stdout:HOST ADDR LABELS STATUS 2023-08-11T15:46:09.332 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 172.21.15.32 2023-08-11T15:46:09.332 INFO:teuthology.orchestra.run.smithi032.stdout:smithi134 172.21.15.134 2023-08-11T15:46:09.333 INFO:teuthology.orchestra.run.smithi032.stdout:smithi192 172.21.15.192 2023-08-11T15:46:09.333 INFO:teuthology.orchestra.run.smithi032.stdout:3 hosts in cluster 2023-08-11T15:46:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81fc09ffe0 msgr2=0x7f81fc0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81fc09ffe0 0x7f81fc0a24a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8208004800 tx=0x7f820801a040 comp rx=0 tx=0).stop 2023-08-11T15:46:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82141009b0 msgr2=0x7f82140746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82141009b0 0x7f82140746c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f820400e9c0 tx=0x7f820400ecd0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 shutdown_connections 2023-08-11T15:46:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f81fc09ffe0 0x7f81fc0a24a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8214102550 0x7f8214073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82141009b0 0x7f82140746c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 --2- 172.21.15.32:0/1920993030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214101bb0 0x7f8214072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 >> 172.21.15.32:0/1920993030 conn(0x7f82140fbf10 msgr2=0x7f82140fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 shutdown_connections 2023-08-11T15:46:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:09.333+0000 7f821a7cf700 1 -- 172.21.15.32:0/1920993030 wait complete. 2023-08-11T15:46:09.481 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:09 smithi032 ceph-mon[97103]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T15:46:09.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:09 smithi192 ceph-mon[130806]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T15:46:09.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:09 smithi134 ceph-mon[130447]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T15:46:09.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- bash -c 'ceph orch device ls' 2023-08-11T15:46:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:46:11.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:11 smithi032 ceph-mon[97103]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:11.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:11 smithi032 ceph-mon[97103]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:11.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:11 smithi192 ceph-mon[130806]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:11.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:11 smithi192 ceph-mon[130806]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:11.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:11 smithi134 ceph-mon[130447]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:11.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:11 smithi134 ceph-mon[130447]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 -- 172.21.15.32:0/2924594771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941009b0 msgr2=0x7f0794100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 --2- 172.21.15.32:0/2924594771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941009b0 0x7f0794100dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f07840099c0 tx=0x7f0784009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 -- 172.21.15.32:0/2924594771 shutdown_connections 2023-08-11T15:46:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 --2- 172.21.15.32:0/2924594771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941009b0 0x7f0794100dd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 --2- 172.21.15.32:0/2924594771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0794102550 0x7f0794108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 --2- 172.21.15.32:0/2924594771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0794101bb0 0x7f0794102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 -- 172.21.15.32:0/2924594771 >> 172.21.15.32:0/2924594771 conn(0x7f07940fbf10 msgr2=0x7f07940fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:11.999+0000 7f0798b7e700 1 -- 172.21.15.32:0/2924594771 shutdown_connections 2023-08-11T15:46:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.000+0000 7f0798b7e700 1 -- 172.21.15.32:0/2924594771 wait complete. 2023-08-11T15:46:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.000+0000 7f0798b7e700 1 Processor -- start 2023-08-11T15:46:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0798b7e700 1 -- start start 2023-08-11T15:46:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0798b7e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07941009b0 0x7f07941988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0798b7e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0794101bb0 0x7f0794198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0798b7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794102550 0x7f079419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0798b7e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07941a0460 con 0x7f0794102550 2023-08-11T15:46:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0798b7e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07941a05a0 con 0x7f07941009b0 2023-08-11T15:46:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0798b7e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07941a0730 con 0x7f0794101bb0 2023-08-11T15:46:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0792d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794102550 0x7f079419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0792d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794102550 0x7f079419fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57726/0 (socket says 172.21.15.32:57726) 2023-08-11T15:46:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0792d9d700 1 -- 172.21.15.32:0/1498369115 learned_addr learned my addr 172.21.15.32:0/1498369115 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f079259c700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07941009b0 0x7f07941988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.001+0000 7f0791d9b700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0794101bb0 0x7f0794198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f0792d9d700 1 -- 172.21.15.32:0/1498369115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0794101bb0 msgr2=0x7f0794198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f0792d9d700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0794101bb0 0x7f0794198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f0792d9d700 1 -- 172.21.15.32:0/1498369115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07941009b0 msgr2=0x7f07941988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f0792d9d700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07941009b0 0x7f07941988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f0792d9d700 1 -- 172.21.15.32:0/1498369115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840095f0 con 0x7f0794102550 2023-08-11T15:46:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f0792d9d700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794102550 0x7f079419fe90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f078c00ef60 tx=0x7f078c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f07837fe700 1 -- 172.21.15.32:0/1498369115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c018070 con 0x7f0794102550 2023-08-11T15:46:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07941a09b0 con 0x7f0794102550 2023-08-11T15:46:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f07837fe700 1 -- 172.21.15.32:0/1498369115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f078c00f040 con 0x7f0794102550 2023-08-11T15:46:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.002+0000 7f07837fe700 1 -- 172.21.15.32:0/1498369115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00ccc0 con 0x7f0794102550 2023-08-11T15:46:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.003+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07941a0eb0 con 0x7f0794102550 2023-08-11T15:46:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.004+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0794192ff0 con 0x7f0794102550 2023-08-11T15:46:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.006+0000 7f07837fe700 1 -- 172.21.15.32:0/1498369115 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f078c003a00 con 0x7f0794102550 2023-08-11T15:46:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.007+0000 7f07837fe700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f077c09ffe0 0x7f077c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.007+0000 7f07837fe700 1 -- 172.21.15.32:0/1498369115 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f078c0cb200 con 0x7f0794102550 2023-08-11T15:46:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.007+0000 7f079259c700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f077c09ffe0 0x7f077c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.008+0000 7f079259c700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f077c09ffe0 0x7f077c0a24a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0784009990 tx=0x7f0784005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.011+0000 7f07837fe700 1 -- 172.21.15.32:0/1498369115 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f078c094b30 con 0x7f0794102550 2023-08-11T15:46:12.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.234+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f079402cdf0 con 0x7f077c09ffe0 2023-08-11T15:46:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.235+0000 7f07837fe700 1 -- 172.21.15.32:0/1498369115 <== mgr.14148 v2:172.21.15.32:6800/2482310260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f079402cdf0 con 0x7f077c09ffe0 2023-08-11T15:46:12.237 INFO:teuthology.orchestra.run.smithi032.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-08-11T15:46:12.237 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402DN480DGN 447G No 3m ago LVM detected, locked 2023-08-11T15:46:12.238 INFO:teuthology.orchestra.run.smithi032.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 3m ago LVM detected, locked 2023-08-11T15:46:12.238 INFO:teuthology.orchestra.run.smithi032.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 2m ago LVM detected, locked 2023-08-11T15:46:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f077c09ffe0 msgr2=0x7f077c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f077c09ffe0 0x7f077c0a24a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0784009990 tx=0x7f0784005fb0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794102550 msgr2=0x7f079419fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794102550 0x7f079419fe90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f078c00ef60 tx=0x7f078c00ef90 comp rx=0 tx=0).stop 2023-08-11T15:46:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 shutdown_connections 2023-08-11T15:46:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f077c09ffe0 0x7f077c0a24a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794102550 0x7f079419fe90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07941009b0 0x7f07941988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 --2- 172.21.15.32:0/1498369115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0794101bb0 0x7f0794198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 >> 172.21.15.32:0/1498369115 conn(0x7f07940fbf10 msgr2=0x7f0794107530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 shutdown_connections 2023-08-11T15:46:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:12.238+0000 7f0798b7e700 1 -- 172.21.15.32:0/1498369115 wait complete. 2023-08-11T15:46:12.683 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-11T15:46:12.693 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:46:12.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- bash -c 'ceph fs dump' 2023-08-11T15:46:13.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:12 smithi032 ceph-mon[97103]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:13.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:12 smithi032 ceph-mon[97103]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:13.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:12 smithi134 ceph-mon[130447]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:13.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:12 smithi134 ceph-mon[130447]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:13.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:12 smithi192 ceph-mon[130806]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:13.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:12 smithi192 ceph-mon[130806]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:13.430 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:46:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 -- 172.21.15.32:0/2459335492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54101bb0 msgr2=0x7fbd54102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2459335492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54101bb0 0x7fbd54102010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbd440099c0 tx=0x7fbd44009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 -- 172.21.15.32:0/2459335492 shutdown_connections 2023-08-11T15:46:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2459335492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd54102550 0x7fbd54108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2459335492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 0x7fbd54100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2459335492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54101bb0 0x7fbd54102010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 -- 172.21.15.32:0/2459335492 >> 172.21.15.32:0/2459335492 conn(0x7fbd540fbf10 msgr2=0x7fbd540fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 -- 172.21.15.32:0/2459335492 shutdown_connections 2023-08-11T15:46:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.943+0000 7fbd5c587700 1 -- 172.21.15.32:0/2459335492 wait complete. 2023-08-11T15:46:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 Processor -- start 2023-08-11T15:46:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 -- start start 2023-08-11T15:46:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 0x7fbd541966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd54101bb0 0x7fbd54196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54102550 0x7fbd5419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5419e270 con 0x7fbd54102550 2023-08-11T15:46:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5419e3b0 con 0x7fbd54101bb0 2023-08-11T15:46:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5c587700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5419e540 con 0x7fbd541009b0 2023-08-11T15:46:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5a323700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 0x7fbd541966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.945+0000 7fbd5a323700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 0x7fbd541966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:33006/0 (socket says 172.21.15.32:33006) 2023-08-11T15:46:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5a323700 1 -- 172.21.15.32:0/2662592079 learned_addr learned my addr 172.21.15.32:0/2662592079 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5ab24700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54102550 0x7fbd5419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd59b22700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd54101bb0 0x7fbd54196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5a323700 1 -- 172.21.15.32:0/2662592079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd54101bb0 msgr2=0x7fbd54196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5a323700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd54101bb0 0x7fbd54196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5a323700 1 -- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54102550 msgr2=0x7fbd5419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5a323700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54102550 0x7fbd5419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5a323700 1 -- 172.21.15.32:0/2662592079 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440095f0 con 0x7fbd541009b0 2023-08-11T15:46:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5a323700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 0x7fbd541966e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000e9a0 tx=0x7fbd5000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd4b7fe700 1 -- 172.21.15.32:0/2662592079 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50009790 con 0x7fbd541009b0 2023-08-11T15:46:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.946+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5419e820 con 0x7fbd541009b0 2023-08-11T15:46:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.947+0000 7fbd4b7fe700 1 -- 172.21.15.32:0/2662592079 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd5000c810 con 0x7fbd541009b0 2023-08-11T15:46:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.947+0000 7fbd4b7fe700 1 -- 172.21.15.32:0/2662592079 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50003c70 con 0x7fbd541009b0 2023-08-11T15:46:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.947+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5419ed70 con 0x7fbd541009b0 2023-08-11T15:46:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.948+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5410c200 con 0x7fbd541009b0 2023-08-11T15:46:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.950+0000 7fbd4b7fe700 1 -- 172.21.15.32:0/2662592079 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fbd5000cc80 con 0x7fbd541009b0 2023-08-11T15:46:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.951+0000 7fbd4b7fe700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbd4009ffe0 0x7fbd400a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.951+0000 7fbd4b7fe700 1 -- 172.21.15.32:0/2662592079 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbd50014070 con 0x7fbd541009b0 2023-08-11T15:46:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.952+0000 7fbd59b22700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbd4009ffe0 0x7fbd400a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.952+0000 7fbd59b22700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbd4009ffe0 0x7fbd400a24a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400b580 tx=0x7fbd44005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:14.953+0000 7fbd4b7fe700 1 -- 172.21.15.32:0/2662592079 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbd50095250 con 0x7fbd541009b0 2023-08-11T15:46:15.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.209+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd5410c410 con 0x7fbd541009b0 2023-08-11T15:46:15.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.210+0000 7fbd4b7fe700 1 -- 172.21.15.32:0/2662592079 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+2917 (secure 0 0 0) 0x7fbd50098870 con 0x7fbd541009b0 2023-08-11T15:46:15.212 INFO:teuthology.orchestra.run.smithi032.stdout:e21 2023-08-11T15:46:15.212 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T15:46:15.212 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T15:46:15.212 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2023-08-11T15:46:15.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:15.212 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2023-08-11T15:46:15.213 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2023-08-11T15:46:15.213 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2023-08-11T15:46:15.213 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-08-11T15:46:15.213 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-08-11T15:44:19.707016+0000 2023-08-11T15:46:15.213 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2023-08-11T15:44:52.172791+0000 2023-08-11T15:46:15.213 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2023-08-11T15:46:15.213 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2023-08-11T15:46:15.214 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 300 2023-08-11T15:46:15.214 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2023-08-11T15:46:15.214 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2023-08-11T15:46:15.214 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2023-08-11T15:46:15.214 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2023-08-11T15:46:15.214 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2023-08-11T15:46:15.214 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=24578} 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2023-08-11T15:46:15.215 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3,4] 2023-08-11T15:46:15.216 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2023-08-11T15:46:15.216 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data disabled 2023-08-11T15:46:15.216 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2023-08-11T15:46:15.216 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2023-08-11T15:46:15.216 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 0 2023-08-11T15:46:15.216 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b{0:24578} state up:active seq 4 addr [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.216 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a{0:24566} state up:standby-replay seq 1 addr [v2:172.21.15.32:6840/1060395589,v1:172.21.15.32:6841/1060395589] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.j{-1:14481} state up:standby seq 1 addr [v2:172.21.15.32:6834/3298780897,v1:172.21.15.32:6835/3298780897] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d{-1:14493} state up:standby seq 1 addr [v2:172.21.15.32:6836/1824615419,v1:172.21.15.32:6837/1824615419] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.g{-1:14535} state up:standby seq 1 addr [v2:172.21.15.32:6838/840045338,v1:172.21.15.32:6839/840045338] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.217 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.l{-1:24460} state up:standby seq 1 addr [v2:172.21.15.192:6832/408140602,v1:172.21.15.192:6833/408140602] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.218 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.k{-1:24467} state up:standby seq 1 addr [v2:172.21.15.134:6832/3616880880,v1:172.21.15.134:6833/3616880880] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.218 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.f{-1:24505} state up:standby seq 1 addr [v2:172.21.15.192:6834/3482775903,v1:172.21.15.192:6835/3482775903] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.218 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.e{-1:24509} state up:standby seq 1 addr [v2:172.21.15.134:6834/1559453954,v1:172.21.15.134:6835/1559453954] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.218 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c{-1:24517} state up:standby seq 1 addr [v2:172.21.15.192:6836/3370727785,v1:172.21.15.192:6837/3370727785] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.218 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.i{-1:24532} state up:standby seq 1 addr [v2:172.21.15.192:6838/2697851094,v1:172.21.15.192:6839/2697851094] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.218 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.h{-1:24533} state up:standby seq 1 addr [v2:172.21.15.134:6836/4150364023,v1:172.21.15.134:6837/4150364023] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T15:46:15.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbd4009ffe0 msgr2=0x7fbd400a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:15.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbd4009ffe0 0x7fbd400a24a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400b580 tx=0x7fbd44005fb0 comp rx=0 tx=0).stop 2023-08-11T15:46:15.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 msgr2=0x7fbd541966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:15.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 0x7fbd541966e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000e9a0 tx=0x7fbd5000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:46:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 shutdown_connections 2023-08-11T15:46:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbd4009ffe0 0x7fbd400a24a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd54102550 0x7fbd5419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd541009b0 0x7fbd541966e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 --2- 172.21.15.32:0/2662592079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd54101bb0 0x7fbd54196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 >> 172.21.15.32:0/2662592079 conn(0x7fbd540fbf10 msgr2=0x7fbd540fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:15.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 shutdown_connections 2023-08-11T15:46:15.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:15.213+0000 7fbd5c587700 1 -- 172.21.15.32:0/2662592079 wait complete. 2023-08-11T15:46:15.221 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-08-11T15:46:15.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:15 smithi134 ceph-mon[130447]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:15.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:15 smithi032 ceph-mon[97103]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:15.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:15 smithi192 ceph-mon[130806]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:15.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- bash -c 'ceph osd dump' 2023-08-11T15:46:16.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:16 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2662592079' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-11T15:46:16.482 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:46:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:16 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2662592079' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-11T15:46:16.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:16 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2662592079' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-11T15:46:17.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:17 smithi032 ceph-mon[97103]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:17.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:17 smithi134 ceph-mon[130447]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:17.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:17 smithi192 ceph-mon[130806]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.025+0000 7f42792ad700 1 -- 172.21.15.32:0/26869443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274102550 msgr2=0x7f4274108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.025+0000 7f42792ad700 1 --2- 172.21.15.32:0/26869443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274102550 0x7f4274108e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f426c00b0d0 tx=0x7f426c00b3e0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.026+0000 7f42792ad700 1 -- 172.21.15.32:0/26869443 shutdown_connections 2023-08-11T15:46:18.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.026+0000 7f42792ad700 1 --2- 172.21.15.32:0/26869443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274102550 0x7f4274108e00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.026+0000 7f42792ad700 1 --2- 172.21.15.32:0/26869443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42741009b0 0x7f4274100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.026+0000 7f42792ad700 1 --2- 172.21.15.32:0/26869443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4274101bb0 0x7f4274102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.026+0000 7f42792ad700 1 -- 172.21.15.32:0/26869443 >> 172.21.15.32:0/26869443 conn(0x7f42740fbf10 msgr2=0x7f42740fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:18.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.026+0000 7f42792ad700 1 -- 172.21.15.32:0/26869443 shutdown_connections 2023-08-11T15:46:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.026+0000 7f42792ad700 1 -- 172.21.15.32:0/26869443 wait complete. 2023-08-11T15:46:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 Processor -- start 2023-08-11T15:46:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 -- start start 2023-08-11T15:46:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42741009b0 0x7f42741989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274101bb0 0x7f4274198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4274102550 0x7f427419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:18.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42741a04f0 con 0x7f4274101bb0 2023-08-11T15:46:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42741a0630 con 0x7f42741009b0 2023-08-11T15:46:18.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.027+0000 7f42792ad700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42741a07c0 con 0x7f4274102550 2023-08-11T15:46:18.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274101bb0 0x7f4274198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274101bb0 0x7f4274198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40192/0 (socket says 172.21.15.32:40192) 2023-08-11T15:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 -- 172.21.15.32:0/3391226428 learned_addr learned my addr 172.21.15.32:0/3391226428 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f4272ffd700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42741009b0 0x7f42741989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42737fe700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4274102550 0x7f427419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 -- 172.21.15.32:0/3391226428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4274102550 msgr2=0x7f427419ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4274102550 0x7f427419ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 -- 172.21.15.32:0/3391226428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42741009b0 msgr2=0x7f42741989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42741009b0 0x7f42741989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 -- 172.21.15.32:0/3391226428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c009d00 con 0x7f4274101bb0 2023-08-11T15:46:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f4272ffd700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42741009b0 0x7f42741989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f42727fc700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274101bb0 0x7f4274198ef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f426800e9d0 tx=0x7f426800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.028+0000 7f425bfff700 1 -- 172.21.15.32:0/3391226428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268009790 con 0x7f4274101bb0 2023-08-11T15:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.029+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42741a0aa0 con 0x7f4274101bb0 2023-08-11T15:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.029+0000 7f425bfff700 1 -- 172.21.15.32:0/3391226428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426800cbd0 con 0x7f4274101bb0 2023-08-11T15:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.029+0000 7f425bfff700 1 -- 172.21.15.32:0/3391226428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268018680 con 0x7f4274101bb0 2023-08-11T15:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.029+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42741a0fa0 con 0x7f4274101bb0 2023-08-11T15:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.030+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4274193000 con 0x7f4274101bb0 2023-08-11T15:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.032+0000 7f425bfff700 1 -- 172.21.15.32:0/3391226428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f426800cd40 con 0x7f4274101bb0 2023-08-11T15:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.033+0000 7f425bfff700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f425c09ffe0 0x7f425c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.033+0000 7f425bfff700 1 -- 172.21.15.32:0/3391226428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4268014070 con 0x7f4274101bb0 2023-08-11T15:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.034+0000 7f4272ffd700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f425c09ffe0 0x7f425c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.034+0000 7f4272ffd700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f425c09ffe0 0x7f425c0a24a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4274101a10 tx=0x7f4264009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.037+0000 7f425bfff700 1 -- 172.21.15.32:0/3391226428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4268051c30 con 0x7f4274101bb0 2023-08-11T15:46:18.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.248+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f427410c210 con 0x7f4274101bb0 2023-08-11T15:46:18.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.249+0000 7f425bfff700 1 -- 172.21.15.32:0/3391226428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5617 (secure 0 0 0) 0x7f4268095270 con 0x7f4274101bb0 2023-08-11T15:46:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 92 2023-08-11T15:46:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:fsid cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T15:46:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-08-11T15:36:56.367133+0000 2023-08-11T15:46:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2023-08-11T15:45:56.189759+0000 2023-08-11T15:46:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-08-11T15:46:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 27 2023-08-11T15:46:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2023-08-11T15:46:18.252 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2023-08-11T15:46:18.252 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2023-08-11T15:46:18.252 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2023-08-11T15:46:18.252 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client luminous 2023-08-11T15:46:18.252 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release reef 2023-08-11T15:46:18.252 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2023-08-11T15:46:18.252 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-08-11T15:46:18.253 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-08-11T15:46:18.253 INFO:teuthology.orchestra.run.smithi032.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-08-11T15:46:18.253 INFO:teuthology.orchestra.run.smithi032.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-08-11T15:46:18.253 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 12 2023-08-11T15:46:18.253 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6802/99848982,v1:172.21.15.32:6803/99848982] [v2:172.21.15.32:6804/99848982,v1:172.21.15.32:6805/99848982] exists,up 880cacc2-d87f-43c6-902e-9dfbab59b3c6 2023-08-11T15:46:18.253 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/251950255,v1:172.21.15.134:6801/251950255] [v2:172.21.15.134:6802/251950255,v1:172.21.15.134:6803/251950255] exists,up 7af16e40-e09e-4ef1-ab51-bd03df34e972 2023-08-11T15:46:18.253 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/2619008811,v1:172.21.15.192:6801/2619008811] [v2:172.21.15.192:6802/2619008811,v1:172.21.15.192:6803/2619008811] exists,up 8861d523-118d-4cd5-b4fa-73fbfced3c76 2023-08-11T15:46:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6810/862642324,v1:172.21.15.32:6811/862642324] [v2:172.21.15.32:6812/862642324,v1:172.21.15.32:6813/862642324] exists,up d001e237-aefd-4118-b7e4-978433e52622 2023-08-11T15:46:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 up in weight 1 up_from 34 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/3721901848,v1:172.21.15.134:6809/3721901848] [v2:172.21.15.134:6810/3721901848,v1:172.21.15.134:6811/3721901848] exists,up 6fc91951-f4cb-4edd-8713-d70f306ba0f5 2023-08-11T15:46:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 up in weight 1 up_from 39 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/3471337783,v1:172.21.15.192:6809/3471337783] [v2:172.21.15.192:6810/3471337783,v1:172.21.15.192:6811/3471337783] exists,up bfb4141a-f600-40c2-a097-ee05482183ac 2023-08-11T15:46:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6818/846261064,v1:172.21.15.32:6819/846261064] [v2:172.21.15.32:6820/846261064,v1:172.21.15.32:6821/846261064] exists,up 8ba2f452-da4c-4d75-9452-fd60cdcc25a3 2023-08-11T15:46:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 up in weight 1 up_from 52 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/4228806294,v1:172.21.15.134:6817/4228806294] [v2:172.21.15.134:6818/4228806294,v1:172.21.15.134:6819/4228806294] exists,up 12bf8d04-54c5-4aa8-ae64-14779ba45002 2023-08-11T15:46:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 up in weight 1 up_from 59 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/2583846672,v1:172.21.15.192:6817/2583846672] [v2:172.21.15.192:6818/2583846672,v1:172.21.15.192:6819/2583846672] exists,up c9797fa6-65ca-40f5-aa93-b4120c4400ce 2023-08-11T15:46:18.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 up in weight 1 up_from 67 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6826/1556490683,v1:172.21.15.32:6827/1556490683] [v2:172.21.15.32:6828/1556490683,v1:172.21.15.32:6829/1556490683] exists,up 5b7e6dc0-e7d9-4003-a922-2764ad27b7c3 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 up in weight 1 up_from 72 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/1937504104,v1:172.21.15.134:6825/1937504104] [v2:172.21.15.134:6826/1937504104,v1:172.21.15.134:6827/1937504104] exists,up 06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 up in weight 1 up_from 79 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/1267688108,v1:172.21.15.192:6825/1267688108] [v2:172.21.15.192:6826/1267688108,v1:172.21.15.192:6827/1267688108] exists,up 2799be08-a3bf-4a40-a968-0e091285d8c6 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.10 [8,11] 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.13 [2,11] 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.16 [2,4] 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.1 [4,9] 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.c [3,9] 2023-08-11T15:46:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.1d [5,9] 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.27 [4,8] 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.2 [9,1] 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.8 [5,10] 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.18 [9,8] 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 4.1b [9,8] 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/33820350 expires 2023-08-12T15:37:53.986517+0000 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/4253467068 expires 2023-08-12T15:37:53.986517+0000 2023-08-11T15:46:18.256 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3287657132 expires 2023-08-12T15:37:53.986517+0000 2023-08-11T15:46:18.257 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3253045891 expires 2023-08-12T15:37:27.338674+0000 2023-08-11T15:46:18.257 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3969802586 expires 2023-08-12T15:37:27.338674+0000 2023-08-11T15:46:18.257 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/114266517 expires 2023-08-12T15:37:27.338674+0000 2023-08-11T15:46:18.257 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/114266517 expires 2023-08-12T15:37:27.338674+0000 2023-08-11T15:46:18.257 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/4253467068 expires 2023-08-12T15:37:53.986517+0000 2023-08-11T15:46:18.257 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3000621007 expires 2023-08-12T15:37:27.338674+0000 2023-08-11T15:46:18.257 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/1572440619 expires 2023-08-12T15:37:53.986517+0000 2023-08-11T15:46:18.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f425c09ffe0 msgr2=0x7f425c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:18.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f425c09ffe0 0x7f425c0a24a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4274101a10 tx=0x7f4264009260 comp rx=0 tx=0).stop 2023-08-11T15:46:18.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274101bb0 msgr2=0x7f4274198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:18.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274101bb0 0x7f4274198ef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f426800e9d0 tx=0x7f426800ece0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 shutdown_connections 2023-08-11T15:46:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f425c09ffe0 0x7f425c0a24a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4274102550 0x7f427419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42741009b0 0x7f42741989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 --2- 172.21.15.32:0/3391226428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4274101bb0 0x7f4274198ef0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 >> 172.21.15.32:0/3391226428 conn(0x7f42740fbf10 msgr2=0x7f4274107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 shutdown_connections 2023-08-11T15:46:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:18.252+0000 7f42792ad700 1 -- 172.21.15.32:0/3391226428 wait complete. 2023-08-11T15:46:18.780 INFO:teuthology.run_tasks:Running task fs.ready... 2023-08-11T15:46:18.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:46:19.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.003+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1726025691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9641024b0 msgr2=0x7fe964108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.003+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1726025691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9641024b0 0x7fe964108d60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00b0d0 tx=0x7fe95c00b3e0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.003+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1726025691 shutdown_connections 2023-08-11T15:46:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.003+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1726025691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9641024b0 0x7fe964108d60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.003+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1726025691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe964100920 0x7fe964100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.003+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1726025691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe964101b10 0x7fe964101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.003+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1726025691 >> 172.21.15.32:0/1726025691 conn(0x7fe9640fbe50 msgr2=0x7fe9640fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.004+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1726025691 shutdown_connections 2023-08-11T15:46:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.004+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1726025691 wait complete. 2023-08-11T15:46:19.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 Processor -- start 2023-08-11T15:46:19.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 -- start start 2023-08-11T15:46:19.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe964100920 0x7fe9641988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe964101b10 0x7fe964198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9641024b0 0x7fe96419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9641a0420 con 0x7fe964100920 2023-08-11T15:46:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9641a0560 con 0x7fe964101b10 2023-08-11T15:46:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe96c1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9641a06f0 con 0x7fe9641024b0 2023-08-11T15:46:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe969f9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe964100920 0x7fe9641988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe969f9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe964100920 0x7fe9641988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40216/0 (socket says 172.21.15.32:40216) 2023-08-11T15:46:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.005+0000 7fe969f9b700 1 -- 172.21.15.32:0/1785263140 learned_addr learned my addr 172.21.15.32:0/1785263140 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe96a79c700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9641024b0 0x7fe96419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe96979a700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe964101b10 0x7fe964198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe969f9b700 1 -- 172.21.15.32:0/1785263140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9641024b0 msgr2=0x7fe96419fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe969f9b700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9641024b0 0x7fe96419fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe969f9b700 1 -- 172.21.15.32:0/1785263140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe964101b10 msgr2=0x7fe964198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe969f9b700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe964101b10 0x7fe964198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe969f9b700 1 -- 172.21.15.32:0/1785263140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c009d00 con 0x7fe964100920 2023-08-11T15:46:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe969f9b700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe964100920 0x7fe9641988a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe96000d980 tx=0x7fe96000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe95affd700 1 -- 172.21.15.32:0/1785263140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000f890 con 0x7fe964100920 2023-08-11T15:46:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe95affd700 1 -- 172.21.15.32:0/1785263140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe960010460 con 0x7fe964100920 2023-08-11T15:46:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe95affd700 1 -- 172.21.15.32:0/1785263140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600195c0 con 0x7fe964100920 2023-08-11T15:46:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.006+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9641a09d0 con 0x7fe964100920 2023-08-11T15:46:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.007+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9641a0e90 con 0x7fe964100920 2023-08-11T15:46:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.007+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96410c140 con 0x7fe964100920 2023-08-11T15:46:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.009+0000 7fe95affd700 1 -- 172.21.15.32:0/1785263140 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fe9600105d0 con 0x7fe964100920 2023-08-11T15:46:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.011+0000 7fe95affd700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe9500a00a0 0x7fe9500a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.011+0000 7fe95affd700 1 -- 172.21.15.32:0/1785263140 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9600cb440 con 0x7fe964100920 2023-08-11T15:46:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.011+0000 7fe96979a700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe9500a00a0 0x7fe9500a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.012+0000 7fe96979a700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe9500a00a0 0x7fe9500a2560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe954009be0 tx=0x7fe954009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.015+0000 7fe95affd700 1 -- 172.21.15.32:0/1785263140 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe960094ce0 con 0x7fe964100920 2023-08-11T15:46:19.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.229+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe96410c380 con 0x7fe964100920 2023-08-11T15:46:19.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.230+0000 7fe95affd700 1 -- 172.21.15.32:0/1785263140 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24508 (secure 0 0 0) 0x7fe960098300 con 0x7fe964100920 2023-08-11T15:46:19.232 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:19.233 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:45:56.189759+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:46:19.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe9500a00a0 msgr2=0x7fe9500a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe9500a00a0 0x7fe9500a2560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe954009be0 tx=0x7fe954009260 comp rx=0 tx=0).stop 2023-08-11T15:46:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe964100920 msgr2=0x7fe9641988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe964100920 0x7fe9641988a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe96000d980 tx=0x7fe96000dc90 comp rx=0 tx=0).stop 2023-08-11T15:46:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 shutdown_connections 2023-08-11T15:46:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fe9500a00a0 0x7fe9500a2560 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe964100920 0x7fe9641988a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9641024b0 0x7fe96419fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 --2- 172.21.15.32:0/1785263140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe964101b10 0x7fe964198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 >> 172.21.15.32:0/1785263140 conn(0x7fe9640fbe50 msgr2=0x7fe964107370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 shutdown_connections 2023-08-11T15:46:19.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.233+0000 7fe96c1ff700 1 -- 172.21.15.32:0/1785263140 wait complete. 2023-08-11T15:46:19.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:46:19.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:46:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.460+0000 7f187d5ff700 1 -- 172.21.15.32:0/3547250557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1878101b00 msgr2=0x7f1878101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.460+0000 7f187d5ff700 1 --2- 172.21.15.32:0/3547250557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1878101b00 0x7f1878101f60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f186c0099c0 tx=0x7f186c009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.460+0000 7f187d5ff700 1 -- 172.21.15.32:0/3547250557 shutdown_connections 2023-08-11T15:46:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.460+0000 7f187d5ff700 1 --2- 172.21.15.32:0/3547250557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18781024a0 0x7f1878108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.460+0000 7f187d5ff700 1 --2- 172.21.15.32:0/3547250557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1878100910 0x7f1878100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.460+0000 7f187d5ff700 1 --2- 172.21.15.32:0/3547250557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1878101b00 0x7f1878101f60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.460+0000 7f187d5ff700 1 -- 172.21.15.32:0/3547250557 >> 172.21.15.32:0/3547250557 conn(0x7f18780fbe60 msgr2=0x7f18780fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.461+0000 7f187d5ff700 1 -- 172.21.15.32:0/3547250557 shutdown_connections 2023-08-11T15:46:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.461+0000 7f187d5ff700 1 -- 172.21.15.32:0/3547250557 wait complete. 2023-08-11T15:46:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 Processor -- start 2023-08-11T15:46:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 -- start start 2023-08-11T15:46:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1878100910 0x7f18781987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1878101b00 0x7f1878198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18781024a0 0x7f187819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781a0370 con 0x7f18781024a0 2023-08-11T15:46:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781a04b0 con 0x7f1878101b00 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f187d5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781a0640 con 0x7f1878100910 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f18777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18781024a0 0x7f187819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1878100910 0x7f18781987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f18777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18781024a0 0x7f187819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40224/0 (socket says 172.21.15.32:40224) 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f18777fe700 1 -- 172.21.15.32:0/1587546246 learned_addr learned my addr 172.21.15.32:0/1587546246 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.462+0000 7f18767fc700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1878101b00 0x7f1878198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f18777fe700 1 -- 172.21.15.32:0/1587546246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1878100910 msgr2=0x7f18781987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f18777fe700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1878100910 0x7f18781987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f18777fe700 1 -- 172.21.15.32:0/1587546246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1878101b00 msgr2=0x7f1878198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f18777fe700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1878101b00 0x7f1878198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f18777fe700 1 -- 172.21.15.32:0/1587546246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0095f0 con 0x7f18781024a0 2023-08-11T15:46:19.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f18777fe700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18781024a0 0x7f187819fda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f187000ef30 tx=0x7f187000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:19.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f185ffff700 1 -- 172.21.15.32:0/1587546246 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870018070 con 0x7f18781024a0 2023-08-11T15:46:19.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18781a08c0 con 0x7f18781024a0 2023-08-11T15:46:19.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f185ffff700 1 -- 172.21.15.32:0/1587546246 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f187000f040 con 0x7f18781024a0 2023-08-11T15:46:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f185ffff700 1 -- 172.21.15.32:0/1587546246 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187000cd00 con 0x7f18781024a0 2023-08-11T15:46:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.463+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18781a0dd0 con 0x7f18781024a0 2023-08-11T15:46:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.464+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1878192f10 con 0x7f18781024a0 2023-08-11T15:46:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.467+0000 7f185ffff700 1 -- 172.21.15.32:0/1587546246 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f1870003a00 con 0x7f18781024a0 2023-08-11T15:46:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.468+0000 7f185ffff700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f186009ffe0 0x7f18600a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.468+0000 7f185ffff700 1 -- 172.21.15.32:0/1587546246 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f18700cb6f0 con 0x7f18781024a0 2023-08-11T15:46:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.468+0000 7f1876ffd700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f186009ffe0 0x7f18600a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.469+0000 7f1876ffd700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f186009ffe0 0x7f18600a24a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1878101960 tx=0x7f1868009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:19.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:19 smithi134 ceph-mon[130447]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:19.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:19 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3391226428' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-11T15:46:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.470+0000 7f185ffff700 1 -- 172.21.15.32:0/1587546246 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1870095020 con 0x7f18781024a0 2023-08-11T15:46:19.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:19 smithi032 ceph-mon[97103]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:19.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:19 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3391226428' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-11T15:46:19.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:19 smithi192 ceph-mon[130806]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:19.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:19 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3391226428' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-11T15:46:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.678+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f187810c120 con 0x7f18781024a0 2023-08-11T15:46:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.678+0000 7f185ffff700 1 -- 172.21.15.32:0/1587546246 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1870098640 con 0x7f18781024a0 2023-08-11T15:46:19.680 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:46:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.680+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f186009ffe0 msgr2=0x7f18600a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.680+0000 7f187d5ff700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f186009ffe0 0x7f18600a24a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1878101960 tx=0x7f1868009260 comp rx=0 tx=0).stop 2023-08-11T15:46:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.680+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18781024a0 msgr2=0x7f187819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.680+0000 7f187d5ff700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18781024a0 0x7f187819fda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f187000ef30 tx=0x7f187000c3a0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 shutdown_connections 2023-08-11T15:46:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f186009ffe0 0x7f18600a24a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18781024a0 0x7f187819fda0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1878100910 0x7f18781987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 --2- 172.21.15.32:0/1587546246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1878101b00 0x7f1878198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 >> 172.21.15.32:0/1587546246 conn(0x7f18780fbe60 msgr2=0x7f1878107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 shutdown_connections 2023-08-11T15:46:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.681+0000 7f187d5ff700 1 -- 172.21.15.32:0/1587546246 wait complete. 2023-08-11T15:46:19.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:46:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 -- 172.21.15.32:0/4051685655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc100930 msgr2=0x7fd9bc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/4051685655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc100930 0x7fd9bc100d50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0099c0 tx=0x7fd9ac009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 -- 172.21.15.32:0/4051685655 shutdown_connections 2023-08-11T15:46:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/4051685655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc1024c0 0x7fd9bc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/4051685655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc100930 0x7fd9bc100d50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/4051685655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9bc101b20 0x7fd9bc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 -- 172.21.15.32:0/4051685655 >> 172.21.15.32:0/4051685655 conn(0x7fd9bc0fbea0 msgr2=0x7fd9bc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 -- 172.21.15.32:0/4051685655 shutdown_connections 2023-08-11T15:46:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.910+0000 7fd9c29ff700 1 -- 172.21.15.32:0/4051685655 wait complete. 2023-08-11T15:46:19.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.912+0000 7fd9c29ff700 1 Processor -- start 2023-08-11T15:46:19.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.912+0000 7fd9c29ff700 1 -- start start 2023-08-11T15:46:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.912+0000 7fd9c29ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc100930 0x7fd9bc196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9c29ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc101b20 0x7fd9bc196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9c29ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9bc1024c0 0x7fd9bc19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9c29ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19e1d0 con 0x7fd9bc100930 2023-08-11T15:46:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9c29ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19e310 con 0x7fd9bc101b20 2023-08-11T15:46:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9c29ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19e4a0 con 0x7fd9bc1024c0 2023-08-11T15:46:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc100930 0x7fd9bc196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc100930 0x7fd9bc196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40250/0 (socket says 172.21.15.32:40250) 2023-08-11T15:46:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 -- 172.21.15.32:0/3604687984 learned_addr learned my addr 172.21.15.32:0/3604687984 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9b3fff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc101b20 0x7fd9bc196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9c0f9c700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9bc1024c0 0x7fd9bc19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 -- 172.21.15.32:0/3604687984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9bc1024c0 msgr2=0x7fd9bc19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9bc1024c0 0x7fd9bc19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 -- 172.21.15.32:0/3604687984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc101b20 msgr2=0x7fd9bc196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc101b20 0x7fd9bc196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 -- 172.21.15.32:0/3604687984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0095f0 con 0x7fd9bc100930 2023-08-11T15:46:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9c0f9c700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9bc1024c0 0x7fd9bc19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:46:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.913+0000 7fd9bbfff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc100930 0x7fd9bc196650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009fd0 tx=0x7fd9ac00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.914+0000 7fd9b9ffb700 1 -- 172.21.15.32:0/3604687984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac01d070 con 0x7fd9bc100930 2023-08-11T15:46:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.914+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc19e720 con 0x7fd9bc100930 2023-08-11T15:46:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.914+0000 7fd9b9ffb700 1 -- 172.21.15.32:0/3604687984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9ac005510 con 0x7fd9bc100930 2023-08-11T15:46:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.914+0000 7fd9b9ffb700 1 -- 172.21.15.32:0/3604687984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac004bb0 con 0x7fd9bc100930 2023-08-11T15:46:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.914+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc19eba0 con 0x7fd9bc100930 2023-08-11T15:46:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.915+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc1078b0 con 0x7fd9bc100930 2023-08-11T15:46:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.917+0000 7fd9b9ffb700 1 -- 172.21.15.32:0/3604687984 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fd9ac004d10 con 0x7fd9bc100930 2023-08-11T15:46:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.918+0000 7fd9b9ffb700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a809ffe0 0x7fd9a80a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.918+0000 7fd9b9ffb700 1 -- 172.21.15.32:0/3604687984 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd9ac0cca60 con 0x7fd9bc100930 2023-08-11T15:46:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.918+0000 7fd9b3fff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a809ffe0 0x7fd9a80a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.919+0000 7fd9b3fff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a809ffe0 0x7fd9a80a24a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009bf0 tx=0x7fd9a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:19.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:19.920+0000 7fd9b9ffb700 1 -- 172.21.15.32:0/3604687984 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd9ac096390 con 0x7fd9bc100930 2023-08-11T15:46:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.130+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9bc10c320 con 0x7fd9bc100930 2023-08-11T15:46:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.131+0000 7fd9b9ffb700 1 -- 172.21.15.32:0/3604687984 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9ac0999b0 con 0x7fd9bc100930 2023-08-11T15:46:20.133 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:46:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a809ffe0 msgr2=0x7fd9a80a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a809ffe0 0x7fd9a80a24a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009bf0 tx=0x7fd9a4009260 comp rx=0 tx=0).stop 2023-08-11T15:46:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc100930 msgr2=0x7fd9bc196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc100930 0x7fd9bc196650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009fd0 tx=0x7fd9ac00f740 comp rx=0 tx=0).stop 2023-08-11T15:46:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 shutdown_connections 2023-08-11T15:46:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd9a809ffe0 0x7fd9a80a24a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9bc100930 0x7fd9bc196650 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9bc1024c0 0x7fd9bc19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 --2- 172.21.15.32:0/3604687984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc101b20 0x7fd9bc196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.133+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 >> 172.21.15.32:0/3604687984 conn(0x7fd9bc0fbea0 msgr2=0x7fd9bc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.134+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 shutdown_connections 2023-08-11T15:46:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.134+0000 7fd9c29ff700 1 -- 172.21.15.32:0/3604687984 wait complete. 2023-08-11T15:46:20.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:46:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 -- 172.21.15.32:0/1473223494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc584100910 msgr2=0x7fc584100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 --2- 172.21.15.32:0/1473223494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc584100910 0x7fc584100d30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc5740099c0 tx=0x7fc574009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 -- 172.21.15.32:0/1473223494 shutdown_connections 2023-08-11T15:46:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 --2- 172.21.15.32:0/1473223494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc584100910 0x7fc584100d30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 --2- 172.21.15.32:0/1473223494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5841024a0 0x7fc584108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 --2- 172.21.15.32:0/1473223494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc584101b00 0x7fc584101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 -- 172.21.15.32:0/1473223494 >> 172.21.15.32:0/1473223494 conn(0x7fc5840fbe60 msgr2=0x7fc5840fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 -- 172.21.15.32:0/1473223494 shutdown_connections 2023-08-11T15:46:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.363+0000 7fc5893ff700 1 -- 172.21.15.32:0/1473223494 wait complete. 2023-08-11T15:46:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 Processor -- start 2023-08-11T15:46:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 -- start start 2023-08-11T15:46:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc584100910 0x7fc5840746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc584101b00 0x7fc584072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5841024a0 0x7fc584073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58419a500 con 0x7fc5841024a0 2023-08-11T15:46:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc584073790 con 0x7fc584101b00 2023-08-11T15:46:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.364+0000 7fc5893ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc584073950 con 0x7fc584100910 2023-08-11T15:46:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5841024a0 0x7fc584073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc584101b00 0x7fc584072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc584101b00 0x7fc584072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:57068/0 (socket says 172.21.15.32:57068) 2023-08-11T15:46:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5841024a0 0x7fc584073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40278/0 (socket says 172.21.15.32:40278) 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 -- 172.21.15.32:0/2203948329 learned_addr learned my addr 172.21.15.32:0/2203948329 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc582ffd700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc584100910 0x7fc5840746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 -- 172.21.15.32:0/2203948329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc584100910 msgr2=0x7fc5840746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc584100910 0x7fc5840746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 -- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5841024a0 msgr2=0x7fc584073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5841024a0 0x7fc584073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 -- 172.21.15.32:0/2203948329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5740095f0 con 0x7fc584101b00 2023-08-11T15:46:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5837fe700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5841024a0 0x7fc584073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.365+0000 7fc5827fc700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc584101b00 0x7fc584072d10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc57800e9a0 tx=0x7fc57800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.366+0000 7fc56bfff700 1 -- 172.21.15.32:0/2203948329 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578009790 con 0x7fc584101b00 2023-08-11T15:46:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.366+0000 7fc56bfff700 1 -- 172.21.15.32:0/2203948329 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc578010430 con 0x7fc584101b00 2023-08-11T15:46:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.366+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5841a4eb0 con 0x7fc584101b00 2023-08-11T15:46:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.366+0000 7fc56bfff700 1 -- 172.21.15.32:0/2203948329 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578018680 con 0x7fc584101b00 2023-08-11T15:46:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.366+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5841a52d0 con 0x7fc584101b00 2023-08-11T15:46:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.367+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58410c170 con 0x7fc584101b00 2023-08-11T15:46:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.369+0000 7fc56bfff700 1 -- 172.21.15.32:0/2203948329 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc5780105a0 con 0x7fc584101b00 2023-08-11T15:46:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.370+0000 7fc56bfff700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc56c0a0060 0x7fc56c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.370+0000 7fc56bfff700 1 -- 172.21.15.32:0/2203948329 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc578014070 con 0x7fc584101b00 2023-08-11T15:46:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.370+0000 7fc582ffd700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc56c0a0060 0x7fc56c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.371+0000 7fc582ffd700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc56c0a0060 0x7fc56c0a2520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc574009990 tx=0x7fc57400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:20.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.374+0000 7fc56bfff700 1 -- 172.21.15.32:0/2203948329 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc578051c30 con 0x7fc584101b00 2023-08-11T15:46:20.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:20 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1785263140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:20 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1587546246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:46:20.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:20 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3604687984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:46:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:20 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1785263140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:20 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1587546246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:46:20.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:20 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3604687984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:46:20.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:20 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1785263140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:20.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:20 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1587546246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:46:20.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:20 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3604687984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:46:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.585+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc584073bd0 con 0x7fc584101b00 2023-08-11T15:46:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.586+0000 7fc56bfff700 1 -- 172.21.15.32:0/2203948329 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc578095140 con 0x7fc584101b00 2023-08-11T15:46:20.589 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:46:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc56c0a0060 msgr2=0x7fc56c0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc56c0a0060 0x7fc56c0a2520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc574009990 tx=0x7fc57400b540 comp rx=0 tx=0).stop 2023-08-11T15:46:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc584101b00 msgr2=0x7fc584072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc584101b00 0x7fc584072d10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc57800e9a0 tx=0x7fc57800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 shutdown_connections 2023-08-11T15:46:20.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc56c0a0060 0x7fc56c0a2520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5841024a0 0x7fc584073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc584100910 0x7fc5840746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 --2- 172.21.15.32:0/2203948329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc584101b00 0x7fc584072d10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 >> 172.21.15.32:0/2203948329 conn(0x7fc5840fbe60 msgr2=0x7fc5840fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.588+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 shutdown_connections 2023-08-11T15:46:20.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.589+0000 7fc5893ff700 1 -- 172.21.15.32:0/2203948329 wait complete. 2023-08-11T15:46:20.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:46:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 -- 172.21.15.32:0/1428779796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 msgr2=0x7fbae0100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 --2- 172.21.15.32:0/1428779796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae0100d30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0099c0 tx=0x7fbadc009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 -- 172.21.15.32:0/1428779796 shutdown_connections 2023-08-11T15:46:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 --2- 172.21.15.32:0/1428779796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae0100d30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 --2- 172.21.15.32:0/1428779796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae01024a0 0x7fbae0108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 --2- 172.21.15.32:0/1428779796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0101b00 0x7fbae0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 -- 172.21.15.32:0/1428779796 >> 172.21.15.32:0/1428779796 conn(0x7fbae00fbe60 msgr2=0x7fbae00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 -- 172.21.15.32:0/1428779796 shutdown_connections 2023-08-11T15:46:20.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.816+0000 7fbae73ff700 1 -- 172.21.15.32:0/1428779796 wait complete. 2023-08-11T15:46:20.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.817+0000 7fbae73ff700 1 Processor -- start 2023-08-11T15:46:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae73ff700 1 -- start start 2023-08-11T15:46:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae73ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae01966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae73ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0101b00 0x7fbae0196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae73ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae01024a0 0x7fbae019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae73ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae019e230 con 0x7fbae0100910 2023-08-11T15:46:20.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae73ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae019e370 con 0x7fbae01024a0 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae73ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae019e500 con 0x7fbae0101b00 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae519b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae01966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae519b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae01966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40290/0 (socket says 172.21.15.32:40290) 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae519b700 1 -- 172.21.15.32:0/3416911059 learned_addr learned my addr 172.21.15.32:0/3416911059 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.818+0000 7fbae499a700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0101b00 0x7fbae0196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae599c700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae01024a0 0x7fbae019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae519b700 1 -- 172.21.15.32:0/3416911059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0101b00 msgr2=0x7fbae0196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae519b700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0101b00 0x7fbae0196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae519b700 1 -- 172.21.15.32:0/3416911059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae01024a0 msgr2=0x7fbae019dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae519b700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae01024a0 0x7fbae019dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae519b700 1 -- 172.21.15.32:0/3416911059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbadc0095f0 con 0x7fbae0100910 2023-08-11T15:46:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae499a700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0101b00 0x7fbae0196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:46:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae519b700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae01966b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009990 tx=0x7fbadc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbad67fc700 1 -- 172.21.15.32:0/3416911059 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc01d070 con 0x7fbae0100910 2023-08-11T15:46:20.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbad67fc700 1 -- 172.21.15.32:0/3416911059 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbadc0054f0 con 0x7fbae0100910 2023-08-11T15:46:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbad67fc700 1 -- 172.21.15.32:0/3416911059 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004c70 con 0x7fbae0100910 2023-08-11T15:46:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae019e780 con 0x7fbae0100910 2023-08-11T15:46:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.819+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae019eba0 con 0x7fbae0100910 2023-08-11T15:46:20.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.821+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae010c170 con 0x7fbae0100910 2023-08-11T15:46:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.822+0000 7fbad67fc700 1 -- 172.21.15.32:0/3416911059 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fbadc00fc30 con 0x7fbae0100910 2023-08-11T15:46:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.825+0000 7fbad67fc700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbacc0a0060 0x7fbacc0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.825+0000 7fbad67fc700 1 -- 172.21.15.32:0/3416911059 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbadc0cc600 con 0x7fbae0100910 2023-08-11T15:46:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.825+0000 7fbae499a700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbacc0a0060 0x7fbacc0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.825+0000 7fbae499a700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbacc0a0060 0x7fbacc0a2520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009cb0 tx=0x7fbad0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:20.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:20.829+0000 7fbad67fc700 1 -- 172.21.15.32:0/3416911059 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbadc095f30 con 0x7fbae0100910 2023-08-11T15:46:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.038+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbae0107890 con 0x7fbae0100910 2023-08-11T15:46:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.039+0000 7fbad67fc700 1 -- 172.21.15.32:0/3416911059 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbadc099550 con 0x7fbae0100910 2023-08-11T15:46:21.041 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:46:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbacc0a0060 msgr2=0x7fbacc0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbacc0a0060 0x7fbacc0a2520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009cb0 tx=0x7fbad0009260 comp rx=0 tx=0).stop 2023-08-11T15:46:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 msgr2=0x7fbae01966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae01966b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009990 tx=0x7fbadc00f740 comp rx=0 tx=0).stop 2023-08-11T15:46:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 shutdown_connections 2023-08-11T15:46:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fbacc0a0060 0x7fbacc0a2520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae0100910 0x7fbae01966b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae01024a0 0x7fbae019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 --2- 172.21.15.32:0/3416911059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0101b00 0x7fbae0196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 >> 172.21.15.32:0/3416911059 conn(0x7fbae00fbe60 msgr2=0x7fbae00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 shutdown_connections 2023-08-11T15:46:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.042+0000 7fbae73ff700 1 -- 172.21.15.32:0/3416911059 wait complete. 2023-08-11T15:46:21.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:46:21.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/3347673369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f28100930 msgr2=0x7f3f28100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/3347673369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f28100930 0x7f3f28100d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f180098f0 tx=0x7f3f18009c00 comp rx=0 tx=0).stop 2023-08-11T15:46:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/3347673369 shutdown_connections 2023-08-11T15:46:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/3347673369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f281024c0 0x7f3f28108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/3347673369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f28100930 0x7f3f28100d50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/3347673369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 0x7f3f28101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/3347673369 >> 172.21.15.32:0/3347673369 conn(0x7f3f280fbe60 msgr2=0x7f3f280fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/3347673369 shutdown_connections 2023-08-11T15:46:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.267+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/3347673369 wait complete. 2023-08-11T15:46:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 Processor -- start 2023-08-11T15:46:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 -- start start 2023-08-11T15:46:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f28100930 0x7f3f281965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 0x7f3f28196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f281024c0 0x7f3f2819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2819e170 con 0x7f3f28101b20 2023-08-11T15:46:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2819e2b0 con 0x7f3f28100930 2023-08-11T15:46:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.268+0000 7f3f2e7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2819e440 con 0x7f3f281024c0 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 0x7f3f28196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 0x7f3f28196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40308/0 (socket says 172.21.15.32:40308) 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 -- 172.21.15.32:0/2690576963 learned_addr learned my addr 172.21.15.32:0/2690576963 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f27fff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f28100930 0x7f3f281965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f2cd9c700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f281024c0 0x7f3f2819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 -- 172.21.15.32:0/2690576963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f281024c0 msgr2=0x7f3f2819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f281024c0 0x7f3f2819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 -- 172.21.15.32:0/2690576963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f28100930 msgr2=0x7f3f281965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f28100930 0x7f3f281965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 -- 172.21.15.32:0/2690576963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0095f0 con 0x7f3f28101b20 2023-08-11T15:46:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f2cd9c700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f281024c0 0x7f3f2819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:46:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.269+0000 7f3f277fe700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 0x7f3f28196b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00ec10 tx=0x7f3f1c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.270+0000 7f3f257fa700 1 -- 172.21.15.32:0/2690576963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c00cd40 con 0x7f3f28101b20 2023-08-11T15:46:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.270+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f18009520 con 0x7f3f28101b20 2023-08-11T15:46:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.270+0000 7f3f257fa700 1 -- 172.21.15.32:0/2690576963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f1c0044e0 con 0x7f3f28101b20 2023-08-11T15:46:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.270+0000 7f3f257fa700 1 -- 172.21.15.32:0/2690576963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c010690 con 0x7f3f28101b20 2023-08-11T15:46:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.270+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f28072f80 con 0x7f3f28101b20 2023-08-11T15:46:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.271+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2810c140 con 0x7f3f28101b20 2023-08-11T15:46:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.273+0000 7f3f257fa700 1 -- 172.21.15.32:0/2690576963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f3f1c003710 con 0x7f3f28101b20 2023-08-11T15:46:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.274+0000 7f3f257fa700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3f100a0020 0x7f3f100a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.275+0000 7f3f257fa700 1 -- 172.21.15.32:0/2690576963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3f1c014070 con 0x7f3f28101b20 2023-08-11T15:46:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.275+0000 7f3f27fff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3f100a0020 0x7f3f100a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.275+0000 7f3f27fff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3f100a0020 0x7f3f100a24e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f180098c0 tx=0x7f3f18019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.276+0000 7f3f257fa700 1 -- 172.21.15.32:0/2690576963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3f1c0956e0 con 0x7f3f28101b20 2023-08-11T15:46:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:21 smithi134 ceph-mon[130447]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:21.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:21 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2203948329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:46:21.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:21 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3416911059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:46:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.534+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f28073200 con 0x7f3f28101b20 2023-08-11T15:46:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.535+0000 7f3f257fa700 1 -- 172.21.15.32:0/2690576963 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f3f1c01f340 con 0x7f3f28101b20 2023-08-11T15:46:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:21.537 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:52.172791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:46:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3f100a0020 msgr2=0x7f3f100a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3f100a0020 0x7f3f100a24e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f180098c0 tx=0x7f3f18019040 comp rx=0 tx=0).stop 2023-08-11T15:46:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 msgr2=0x7f3f28196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 0x7f3f28196b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00ec10 tx=0x7f3f1c00ef20 comp rx=0 tx=0).stop 2023-08-11T15:46:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 shutdown_connections 2023-08-11T15:46:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3f100a0020 0x7f3f100a24e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f281024c0 0x7f3f2819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f28100930 0x7f3f281965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 --2- 172.21.15.32:0/2690576963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f28101b20 0x7f3f28196b30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.537+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 >> 172.21.15.32:0/2690576963 conn(0x7f3f280fbe60 msgr2=0x7f3f280fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.538+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 shutdown_connections 2023-08-11T15:46:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.538+0000 7f3f2e7ff700 1 -- 172.21.15.32:0/2690576963 wait complete. 2023-08-11T15:46:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-08-11T15:46:21.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:21 smithi032 ceph-mon[97103]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:21.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:21 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2203948329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:46:21.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:21 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3416911059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:46:21.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:46:21.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:21 smithi192 ceph-mon[130806]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:21 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2203948329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:46:21.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:21 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3416911059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:46:21.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.761+0000 7f31f47ff700 1 -- 172.21.15.32:0/1622575273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec100930 msgr2=0x7f31ec100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.761+0000 7f31f47ff700 1 --2- 172.21.15.32:0/1622575273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec100930 0x7f31ec100d50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f31e80099c0 tx=0x7f31e8009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.762+0000 7f31f47ff700 1 -- 172.21.15.32:0/1622575273 shutdown_connections 2023-08-11T15:46:21.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.762+0000 7f31f47ff700 1 --2- 172.21.15.32:0/1622575273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec100930 0x7f31ec100d50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.762+0000 7f31f47ff700 1 --2- 172.21.15.32:0/1622575273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31ec1024c0 0x7f31ec108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.762+0000 7f31f47ff700 1 --2- 172.21.15.32:0/1622575273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31ec101b20 0x7f31ec101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.762+0000 7f31f47ff700 1 -- 172.21.15.32:0/1622575273 >> 172.21.15.32:0/1622575273 conn(0x7f31ec0fbe60 msgr2=0x7f31ec0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.762+0000 7f31f47ff700 1 -- 172.21.15.32:0/1622575273 shutdown_connections 2023-08-11T15:46:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.762+0000 7f31f47ff700 1 -- 172.21.15.32:0/1622575273 wait complete. 2023-08-11T15:46:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 Processor -- start 2023-08-11T15:46:21.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 -- start start 2023-08-11T15:46:21.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31ec100930 0x7f31ec0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31ec101b20 0x7f31ec072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec1024c0 0x7f31ec073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0739f0 con 0x7f31ec1024c0 2023-08-11T15:46:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec073b60 con 0x7f31ec101b20 2023-08-11T15:46:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f47ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1a0650 con 0x7f31ec100930 2023-08-11T15:46:21.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f2d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec1024c0 0x7f31ec073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f2d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec1024c0 0x7f31ec073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40326/0 (socket says 172.21.15.32:40326) 2023-08-11T15:46:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f2d9c700 1 -- 172.21.15.32:0/3594980049 learned_addr learned my addr 172.21.15.32:0/3594980049 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f259b700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31ec100930 0x7f31ec0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.763+0000 7f31f1d9a700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31ec101b20 0x7f31ec072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31f2d9c700 1 -- 172.21.15.32:0/3594980049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31ec100930 msgr2=0x7f31ec0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31f2d9c700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31ec100930 0x7f31ec0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31f2d9c700 1 -- 172.21.15.32:0/3594980049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31ec101b20 msgr2=0x7f31ec072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31f2d9c700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31ec101b20 0x7f31ec072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31f2d9c700 1 -- 172.21.15.32:0/3594980049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e80095f0 con 0x7f31ec1024c0 2023-08-11T15:46:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31f2d9c700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec1024c0 0x7f31ec073250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f31e4007ee0 tx=0x7f31e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31e37fe700 1 -- 172.21.15.32:0/3594980049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4017070 con 0x7f31ec1024c0 2023-08-11T15:46:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31e37fe700 1 -- 172.21.15.32:0/3594980049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31e400f040 con 0x7f31ec1024c0 2023-08-11T15:46:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec1a0930 con 0x7f31ec1024c0 2023-08-11T15:46:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.764+0000 7f31e37fe700 1 -- 172.21.15.32:0/3594980049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4005580 con 0x7f31ec1024c0 2023-08-11T15:46:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.765+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec1a0df0 con 0x7f31ec1024c0 2023-08-11T15:46:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.765+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec10c190 con 0x7f31ec1024c0 2023-08-11T15:46:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.767+0000 7f31e37fe700 1 -- 172.21.15.32:0/3594980049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f31e400db20 con 0x7f31ec1024c0 2023-08-11T15:46:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.770+0000 7f31e37fe700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31d80a0020 0x7f31d80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.770+0000 7f31e37fe700 1 -- 172.21.15.32:0/3594980049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f31e40caee0 con 0x7f31ec1024c0 2023-08-11T15:46:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.770+0000 7f31f259b700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31d80a0020 0x7f31d80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.771+0000 7f31f259b700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31d80a0020 0x7f31d80a24e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009990 tx=0x7f31e8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.774+0000 7f31e37fe700 1 -- 172.21.15.32:0/3594980049 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f31e4094780 con 0x7f31ec1024c0 2023-08-11T15:46:21.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.985+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31ec10c380 con 0x7f31ec1024c0 2023-08-11T15:46:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.987+0000 7f31e37fe700 1 -- 172.21.15.32:0/3594980049 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24508 (secure 0 0 0) 0x7f31e4097da0 con 0x7f31ec1024c0 2023-08-11T15:46:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:21.991 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:45:56.189759+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:46:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31d80a0020 msgr2=0x7f31d80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31d80a0020 0x7f31d80a24e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009990 tx=0x7f31e8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec1024c0 msgr2=0x7f31ec073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec1024c0 0x7f31ec073250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f31e4007ee0 tx=0x7f31e400d3b0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 shutdown_connections 2023-08-11T15:46:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f31d80a0020 0x7f31d80a24e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31ec1024c0 0x7f31ec073250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31ec100930 0x7f31ec0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 --2- 172.21.15.32:0/3594980049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31ec101b20 0x7f31ec072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 >> 172.21.15.32:0/3594980049 conn(0x7f31ec0fbe60 msgr2=0x7f31ec0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 shutdown_connections 2023-08-11T15:46:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:21.989+0000 7f31f47ff700 1 -- 172.21.15.32:0/3594980049 wait complete. 2023-08-11T15:46:22.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:46:22.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:46:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.219+0000 7f2699fff700 1 -- 172.21.15.32:0/406861315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2694100910 msgr2=0x7f2694100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.219+0000 7f2699fff700 1 --2- 172.21.15.32:0/406861315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2694100910 0x7f2694100d30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f26840099c0 tx=0x7f2684009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.220+0000 7f2699fff700 1 -- 172.21.15.32:0/406861315 shutdown_connections 2023-08-11T15:46:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.220+0000 7f2699fff700 1 --2- 172.21.15.32:0/406861315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2694100910 0x7f2694100d30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.220+0000 7f2699fff700 1 --2- 172.21.15.32:0/406861315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26941024a0 0x7f2694108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.220+0000 7f2699fff700 1 --2- 172.21.15.32:0/406861315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2694101b00 0x7f2694101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.220+0000 7f2699fff700 1 -- 172.21.15.32:0/406861315 >> 172.21.15.32:0/406861315 conn(0x7f26940fbe60 msgr2=0x7f26940fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.220+0000 7f2699fff700 1 -- 172.21.15.32:0/406861315 shutdown_connections 2023-08-11T15:46:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.220+0000 7f2699fff700 1 -- 172.21.15.32:0/406861315 wait complete. 2023-08-11T15:46:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.221+0000 7f2699fff700 1 Processor -- start 2023-08-11T15:46:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.221+0000 7f2699fff700 1 -- start start 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2699fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2694100910 0x7f2694196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2699fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2694101b00 0x7f2694196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2699fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26941024a0 0x7f269419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2699fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419e1b0 con 0x7f26941024a0 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2699fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419e2f0 con 0x7f2694101b00 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2699fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269419e480 con 0x7f2694100910 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2692ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2694101b00 0x7f2694196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2692ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2694101b00 0x7f2694196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:57138/0 (socket says 172.21.15.32:57138) 2023-08-11T15:46:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2692ffd700 1 -- 172.21.15.32:0/1021424770 learned_addr learned my addr 172.21.15.32:0/1021424770 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f26937fe700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2694100910 0x7f2694196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.222+0000 7f2693fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26941024a0 0x7f269419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f26937fe700 1 -- 172.21.15.32:0/1021424770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2694101b00 msgr2=0x7f2694196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f26937fe700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2694101b00 0x7f2694196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f26937fe700 1 -- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26941024a0 msgr2=0x7f269419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f26937fe700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26941024a0 0x7f269419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f26937fe700 1 -- 172.21.15.32:0/1021424770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840095f0 con 0x7f2694100910 2023-08-11T15:46:22.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f2693fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26941024a0 0x7f269419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:22.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f26937fe700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2694100910 0x7f2694196630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26840058e0 tx=0x7f268400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:22.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f2690ff9700 1 -- 172.21.15.32:0/1021424770 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268401d070 con 0x7f2694100910 2023-08-11T15:46:22.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269419e700 con 0x7f2694100910 2023-08-11T15:46:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f2690ff9700 1 -- 172.21.15.32:0/1021424770 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2684005020 con 0x7f2694100910 2023-08-11T15:46:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f2690ff9700 1 -- 172.21.15.32:0/1021424770 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684004a50 con 0x7f2694100910 2023-08-11T15:46:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.223+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269419eac0 con 0x7f2694100910 2023-08-11T15:46:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.224+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269410c170 con 0x7f2694100910 2023-08-11T15:46:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.226+0000 7f2690ff9700 1 -- 172.21.15.32:0/1021424770 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f2684005400 con 0x7f2694100910 2023-08-11T15:46:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.228+0000 7f2690ff9700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f267c09ffe0 0x7f267c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.228+0000 7f2690ff9700 1 -- 172.21.15.32:0/1021424770 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f26840cc300 con 0x7f2694100910 2023-08-11T15:46:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.228+0000 7f2692ffd700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f267c09ffe0 0x7f267c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.228+0000 7f2692ffd700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f267c09ffe0 0x7f267c0a24a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2688007d10 tx=0x7f2688007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.230+0000 7f2690ff9700 1 -- 172.21.15.32:0/1021424770 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2684095c30 con 0x7f2694100910 2023-08-11T15:46:22.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:22 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2690576963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:46:22.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:22 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3594980049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:22.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.441+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2694107840 con 0x7f2694100910 2023-08-11T15:46:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.442+0000 7f2690ff9700 1 -- 172.21.15.32:0/1021424770 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2684005190 con 0x7f2694100910 2023-08-11T15:46:22.443 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:46:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f267c09ffe0 msgr2=0x7f267c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f267c09ffe0 0x7f267c0a24a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2688007d10 tx=0x7f2688007480 comp rx=0 tx=0).stop 2023-08-11T15:46:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2694100910 msgr2=0x7f2694196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2694100910 0x7f2694196630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26840058e0 tx=0x7f268400f690 comp rx=0 tx=0).stop 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 shutdown_connections 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f267c09ffe0 0x7f267c0a24a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26941024a0 0x7f269419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2694100910 0x7f2694196630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 --2- 172.21.15.32:0/1021424770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2694101b00 0x7f2694196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 >> 172.21.15.32:0/1021424770 conn(0x7f26940fbe60 msgr2=0x7f26940fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 shutdown_connections 2023-08-11T15:46:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.444+0000 7f2699fff700 1 -- 172.21.15.32:0/1021424770 wait complete. 2023-08-11T15:46:22.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:46:22.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:22 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2690576963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:46:22.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:22 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3594980049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:22.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:22 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2690576963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:46:22.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:22 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3594980049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.671+0000 7f07f27ff700 1 -- 172.21.15.32:0/4172221209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec101b00 msgr2=0x7f07ec101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.671+0000 7f07f27ff700 1 --2- 172.21.15.32:0/4172221209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec101b00 0x7f07ec101f60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f07e00099c0 tx=0x7f07e0009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.671+0000 7f07f27ff700 1 -- 172.21.15.32:0/4172221209 shutdown_connections 2023-08-11T15:46:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.671+0000 7f07f27ff700 1 --2- 172.21.15.32:0/4172221209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ec1024a0 0x7f07ec108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.671+0000 7f07f27ff700 1 --2- 172.21.15.32:0/4172221209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 0x7f07ec100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.671+0000 7f07f27ff700 1 --2- 172.21.15.32:0/4172221209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec101b00 0x7f07ec101f60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.672+0000 7f07f27ff700 1 -- 172.21.15.32:0/4172221209 >> 172.21.15.32:0/4172221209 conn(0x7f07ec0fbe60 msgr2=0x7f07ec0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.672+0000 7f07f27ff700 1 -- 172.21.15.32:0/4172221209 shutdown_connections 2023-08-11T15:46:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.672+0000 7f07f27ff700 1 -- 172.21.15.32:0/4172221209 wait complete. 2023-08-11T15:46:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 Processor -- start 2023-08-11T15:46:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 -- start start 2023-08-11T15:46:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 0x7f07ec0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ec101b00 0x7f07ec072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec1024a0 0x7f07ec073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0739f0 con 0x7f07ec1024a0 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec073b60 con 0x7f07ec100910 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f27ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1a06c0 con 0x7f07ec101b00 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07f0d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec1024a0 0x7f07ec073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.673+0000 7f07ebfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 0x7f07ec0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 0x7f07ec0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:57148/0 (socket says 172.21.15.32:57148) 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 -- 172.21.15.32:0/451040286 learned_addr learned my addr 172.21.15.32:0/451040286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07f0d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec1024a0 0x7f07ec073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40368/0 (socket says 172.21.15.32:40368) 2023-08-11T15:46:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07eb7fe700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ec101b00 0x7f07ec072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 -- 172.21.15.32:0/451040286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ec101b00 msgr2=0x7f07ec072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ec101b00 0x7f07ec072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 -- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec1024a0 msgr2=0x7f07ec073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec1024a0 0x7f07ec073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 -- 172.21.15.32:0/451040286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00095f0 con 0x7f07ec100910 2023-08-11T15:46:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07eb7fe700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ec101b00 0x7f07ec072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:46:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07f0d9c700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec1024a0 0x7f07ec073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07ebfff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 0x7f07ec0746d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00e970 tx=0x7f07dc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:22.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07e97fa700 1 -- 172.21.15.32:0/451040286 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc009790 con 0x7f07ec100910 2023-08-11T15:46:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec1a09a0 con 0x7f07ec100910 2023-08-11T15:46:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07e97fa700 1 -- 172.21.15.32:0/451040286 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07dc00eec0 con 0x7f07ec100910 2023-08-11T15:46:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.674+0000 7f07e97fa700 1 -- 172.21.15.32:0/451040286 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc003cf0 con 0x7f07ec100910 2023-08-11T15:46:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.675+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec1a0eb0 con 0x7f07ec100910 2023-08-11T15:46:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.676+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec107890 con 0x7f07ec100910 2023-08-11T15:46:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.678+0000 7f07e97fa700 1 -- 172.21.15.32:0/451040286 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f07dc00ccd0 con 0x7f07ec100910 2023-08-11T15:46:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.679+0000 7f07e97fa700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f07d409ffe0 0x7f07d40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.679+0000 7f07e97fa700 1 -- 172.21.15.32:0/451040286 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07dc014070 con 0x7f07ec100910 2023-08-11T15:46:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.679+0000 7f07eb7fe700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f07d409ffe0 0x7f07d40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.680+0000 7f07eb7fe700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f07d409ffe0 0x7f07d40a24a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f07e000b580 tx=0x7f07e001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.681+0000 7f07e97fa700 1 -- 172.21.15.32:0/451040286 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f07dc095380 con 0x7f07ec100910 2023-08-11T15:46:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.899+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07ec02cc20 con 0x7f07ec100910 2023-08-11T15:46:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.900+0000 7f07e97fa700 1 -- 172.21.15.32:0/451040286 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07dc0989a0 con 0x7f07ec100910 2023-08-11T15:46:22.902 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:46:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f07d409ffe0 msgr2=0x7f07d40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f07d409ffe0 0x7f07d40a24a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f07e000b580 tx=0x7f07e001a040 comp rx=0 tx=0).stop 2023-08-11T15:46:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 msgr2=0x7f07ec0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 0x7f07ec0746d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00e970 tx=0x7f07dc00ec80 comp rx=0 tx=0).stop 2023-08-11T15:46:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 shutdown_connections 2023-08-11T15:46:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f07d409ffe0 0x7f07d40a24a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07ec1024a0 0x7f07ec073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07ec100910 0x7f07ec0746d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 --2- 172.21.15.32:0/451040286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ec101b00 0x7f07ec072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 >> 172.21.15.32:0/451040286 conn(0x7f07ec0fbe60 msgr2=0x7f07ec0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 shutdown_connections 2023-08-11T15:46:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:22.903+0000 7f07f27ff700 1 -- 172.21.15.32:0/451040286 wait complete. 2023-08-11T15:46:22.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:46:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 -- 172.21.15.32:0/3418715607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 msgr2=0x7f3ef0108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/3418715607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef0108d50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800b0d0 tx=0x7f3ee800b3e0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 -- 172.21.15.32:0/3418715607 shutdown_connections 2023-08-11T15:46:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/3418715607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef0108d50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/3418715607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ef0100910 0x7f3ef0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/3418715607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef0101b00 0x7f3ef0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 -- 172.21.15.32:0/3418715607 >> 172.21.15.32:0/3418715607 conn(0x7f3ef00fbe60 msgr2=0x7f3ef00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 -- 172.21.15.32:0/3418715607 shutdown_connections 2023-08-11T15:46:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.130+0000 7f3ef73ff700 1 -- 172.21.15.32:0/3418715607 wait complete. 2023-08-11T15:46:23.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.131+0000 7f3ef73ff700 1 Processor -- start 2023-08-11T15:46:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef73ff700 1 -- start start 2023-08-11T15:46:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef73ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef0100910 0x7f3ef0196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef73ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ef0101b00 0x7f3ef0196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef73ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef73ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef019e2b0 con 0x7f3ef01024a0 2023-08-11T15:46:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef73ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef019e3f0 con 0x7f3ef0100910 2023-08-11T15:46:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef73ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef019e580 con 0x7f3ef0101b00 2023-08-11T15:46:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef599c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef599c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40374/0 (socket says 172.21.15.32:40374) 2023-08-11T15:46:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.132+0000 7f3ef599c700 1 -- 172.21.15.32:0/877816265 learned_addr learned my addr 172.21.15.32:0/877816265 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef519b700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef0100910 0x7f3ef0196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef499a700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ef0101b00 0x7f3ef0196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef599c700 1 -- 172.21.15.32:0/877816265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ef0101b00 msgr2=0x7f3ef0196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef599c700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ef0101b00 0x7f3ef0196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef599c700 1 -- 172.21.15.32:0/877816265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef0100910 msgr2=0x7f3ef0196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef599c700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef0100910 0x7f3ef0196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef599c700 1 -- 172.21.15.32:0/877816265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8009d00 con 0x7f3ef01024a0 2023-08-11T15:46:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef499a700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ef0101b00 0x7f3ef0196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:46:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef599c700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef019dce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8007990 tx=0x7f3ee8008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ee67fc700 1 -- 172.21.15.32:0/877816265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800e040 con 0x7f3ef01024a0 2023-08-11T15:46:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef019e800 con 0x7f3ef01024a0 2023-08-11T15:46:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ee67fc700 1 -- 172.21.15.32:0/877816265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee800be30 con 0x7f3ef01024a0 2023-08-11T15:46:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.133+0000 7f3ee67fc700 1 -- 172.21.15.32:0/877816265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee801d6d0 con 0x7f3ef01024a0 2023-08-11T15:46:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.134+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef019ecd0 con 0x7f3ef01024a0 2023-08-11T15:46:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.135+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef010c180 con 0x7f3ef01024a0 2023-08-11T15:46:23.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.138+0000 7f3ee67fc700 1 -- 172.21.15.32:0/877816265 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f3ee80051b0 con 0x7f3ef01024a0 2023-08-11T15:46:23.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.139+0000 7f3ee67fc700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3edc09ffe0 0x7f3edc0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.140+0000 7f3ee67fc700 1 -- 172.21.15.32:0/877816265 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ee80cce80 con 0x7f3ef01024a0 2023-08-11T15:46:23.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.140+0000 7f3ef519b700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3edc09ffe0 0x7f3edc0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.140+0000 7f3ef519b700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3edc09ffe0 0x7f3edc0a24a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0101960 tx=0x7f3eec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:23.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.142+0000 7f3ee67fc700 1 -- 172.21.15.32:0/877816265 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3ee80967b0 con 0x7f3ef01024a0 2023-08-11T15:46:23.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.355+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ef01077f0 con 0x7f3ef01024a0 2023-08-11T15:46:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.355+0000 7f3ee67fc700 1 -- 172.21.15.32:0/877816265 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ee8011390 con 0x7f3ef01024a0 2023-08-11T15:46:23.358 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:46:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3edc09ffe0 msgr2=0x7f3edc0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3edc09ffe0 0x7f3edc0a24a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0101960 tx=0x7f3eec009260 comp rx=0 tx=0).stop 2023-08-11T15:46:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 msgr2=0x7f3ef019dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef019dce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8007990 tx=0x7f3ee8008eb0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 shutdown_connections 2023-08-11T15:46:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3edc09ffe0 0x7f3edc0a24a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef01024a0 0x7f3ef019dce0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef0100910 0x7f3ef0196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 --2- 172.21.15.32:0/877816265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ef0101b00 0x7f3ef0196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 >> 172.21.15.32:0/877816265 conn(0x7f3ef00fbe60 msgr2=0x7f3ef00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 shutdown_connections 2023-08-11T15:46:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.358+0000 7f3ef73ff700 1 -- 172.21.15.32:0/877816265 wait complete. 2023-08-11T15:46:23.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:46:23.440 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:23 smithi032 ceph-mon[97103]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:23.441 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:23 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1021424770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:46:23.441 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:23 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/451040286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:46:23.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:23 smithi134 ceph-mon[130447]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:23.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:23 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1021424770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:46:23.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:23 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/451040286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:46:23.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:23 smithi192 ceph-mon[130806]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:23.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:23 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1021424770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:46:23.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:23 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/451040286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:46:23.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 -- 172.21.15.32:0/436124281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7058103cc0 msgr2=0x7f70581060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 --2- 172.21.15.32:0/436124281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7058103cc0 0x7f70581060b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f704c0098f0 tx=0x7f704c009c00 comp rx=0 tx=0).stop 2023-08-11T15:46:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 -- 172.21.15.32:0/436124281 shutdown_connections 2023-08-11T15:46:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 --2- 172.21.15.32:0/436124281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70581065f0 0x7f7058108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 --2- 172.21.15.32:0/436124281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058101390 0x7f7058103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 --2- 172.21.15.32:0/436124281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7058103cc0 0x7f70581060b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 -- 172.21.15.32:0/436124281 >> 172.21.15.32:0/436124281 conn(0x7f70580fad80 msgr2=0x7f70580fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 -- 172.21.15.32:0/436124281 shutdown_connections 2023-08-11T15:46:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.585+0000 7f705e9ff700 1 -- 172.21.15.32:0/436124281 wait complete. 2023-08-11T15:46:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.586+0000 7f705e9ff700 1 Processor -- start 2023-08-11T15:46:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.586+0000 7f705e9ff700 1 -- start start 2023-08-11T15:46:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f705e9ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7058101390 0x7f7058100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f705e9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058103cc0 0x7f70580ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f705e9ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70581065f0 0x7f70580ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f705e9ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7058100020 con 0x7f7058101390 2023-08-11T15:46:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f705e9ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7058100190 con 0x7f70581065f0 2023-08-11T15:46:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f705e9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70581a06b0 con 0x7f7058103cc0 2023-08-11T15:46:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f7057fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7058101390 0x7f7058100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058103cc0 0x7f70580ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058103cc0 0x7f70580ff340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:33230/0 (socket says 172.21.15.32:33230) 2023-08-11T15:46:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 -- 172.21.15.32:0/2231692968 learned_addr learned my addr 172.21.15.32:0/2231692968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f705cf9c700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70581065f0 0x7f70580ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 -- 172.21.15.32:0/2231692968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70581065f0 msgr2=0x7f70580ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70581065f0 0x7f70580ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 -- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7058101390 msgr2=0x7f7058100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7058101390 0x7f7058100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.587+0000 7f70577fe700 1 -- 172.21.15.32:0/2231692968 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c009520 con 0x7f7058103cc0 2023-08-11T15:46:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.588+0000 7f70577fe700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058103cc0 0x7f70580ff340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f704c009fd0 tx=0x7f704c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.588+0000 7f70557fa700 1 -- 172.21.15.32:0/2231692968 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c01c070 con 0x7f7058103cc0 2023-08-11T15:46:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.588+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70581a0930 con 0x7f7058103cc0 2023-08-11T15:46:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.588+0000 7f70557fa700 1 -- 172.21.15.32:0/2231692968 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f704c00f7d0 con 0x7f7058103cc0 2023-08-11T15:46:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.588+0000 7f70557fa700 1 -- 172.21.15.32:0/2231692968 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c004ad0 con 0x7f7058103cc0 2023-08-11T15:46:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.588+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70581a0d60 con 0x7f7058103cc0 2023-08-11T15:46:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.589+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70580fc900 con 0x7f7058103cc0 2023-08-11T15:46:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.591+0000 7f70557fa700 1 -- 172.21.15.32:0/2231692968 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f704c020070 con 0x7f7058103cc0 2023-08-11T15:46:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.593+0000 7f70557fa700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f704009ffe0 0x7f70400a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.593+0000 7f70557fa700 1 -- 172.21.15.32:0/2231692968 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f704c0cc540 con 0x7f7058103cc0 2023-08-11T15:46:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.593+0000 7f7057fff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f704009ffe0 0x7f70400a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.593+0000 7f7057fff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f704009ffe0 0x7f70400a24a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7048009c60 tx=0x7f7048009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.595+0000 7f70557fa700 1 -- 172.21.15.32:0/2231692968 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f704c095e70 con 0x7f7058103cc0 2023-08-11T15:46:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.805+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f705810c2e0 con 0x7f7058103cc0 2023-08-11T15:46:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.806+0000 7f70557fa700 1 -- 172.21.15.32:0/2231692968 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f704c099490 con 0x7f7058103cc0 2023-08-11T15:46:23.807 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:46:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f704009ffe0 msgr2=0x7f70400a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f704009ffe0 0x7f70400a24a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7048009c60 tx=0x7f7048009260 comp rx=0 tx=0).stop 2023-08-11T15:46:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058103cc0 msgr2=0x7f70580ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058103cc0 0x7f70580ff340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f704c009fd0 tx=0x7f704c00b790 comp rx=0 tx=0).stop 2023-08-11T15:46:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 shutdown_connections 2023-08-11T15:46:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f704009ffe0 0x7f70400a24a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7058101390 0x7f7058100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70581065f0 0x7f70580ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 --2- 172.21.15.32:0/2231692968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7058103cc0 0x7f70580ff340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 >> 172.21.15.32:0/2231692968 conn(0x7f70580fad80 msgr2=0x7f7058102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 shutdown_connections 2023-08-11T15:46:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:23.808+0000 7f705e9ff700 1 -- 172.21.15.32:0/2231692968 wait complete. 2023-08-11T15:46:23.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:46:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 -- 172.21.15.32:0/3994774920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 msgr2=0x7fc518106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3994774920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 0x7fc518106100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009990 tx=0x7fc50c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 -- 172.21.15.32:0/3994774920 shutdown_connections 2023-08-11T15:46:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3994774920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc518106640 0x7fc518108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3994774920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5181013e0 0x7fc5181037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3994774920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 0x7fc518106100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 -- 172.21.15.32:0/3994774920 >> 172.21.15.32:0/3994774920 conn(0x7fc5180fad50 msgr2=0x7fc5180fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 -- 172.21.15.32:0/3994774920 shutdown_connections 2023-08-11T15:46:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.032+0000 7fc51cbff700 1 -- 172.21.15.32:0/3994774920 wait complete. 2023-08-11T15:46:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.033+0000 7fc51cbff700 1 Processor -- start 2023-08-11T15:46:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.033+0000 7fc51cbff700 1 -- start start 2023-08-11T15:46:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51cbff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5181013e0 0x7fc518198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51cbff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 0x7fc518198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51cbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc518106640 0x7fc51819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51cbff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181a0380 con 0x7fc5181013e0 2023-08-11T15:46:24.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51cbff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181a04c0 con 0x7fc518103d10 2023-08-11T15:46:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51cbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181a0650 con 0x7fc518106640 2023-08-11T15:46:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5181013e0 0x7fc518198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5181013e0 0x7fc518198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40416/0 (socket says 172.21.15.32:40416) 2023-08-11T15:46:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 -- 172.21.15.32:0/3126680830 learned_addr learned my addr 172.21.15.32:0/3126680830 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc515d9b700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 0x7fc518198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc516d9d700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc518106640 0x7fc51819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 -- 172.21.15.32:0/3126680830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc518106640 msgr2=0x7fc51819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc518106640 0x7fc51819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 -- 172.21.15.32:0/3126680830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 msgr2=0x7fc518198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 0x7fc518198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc51659c700 1 -- 172.21.15.32:0/3126680830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0095f0 con 0x7fc5181013e0 2023-08-11T15:46:24.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.034+0000 7fc515d9b700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 0x7fc518198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:46:24.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.035+0000 7fc51659c700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5181013e0 0x7fc518198800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc50800e9a0 tx=0x7fc50800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:24.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.035+0000 7fc5077fe700 1 -- 172.21.15.32:0/3126680830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508009790 con 0x7fc5181013e0 2023-08-11T15:46:24.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.035+0000 7fc5077fe700 1 -- 172.21.15.32:0/3126680830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc50800cbd0 con 0x7fc5181013e0 2023-08-11T15:46:24.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.035+0000 7fc5077fe700 1 -- 172.21.15.32:0/3126680830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508018680 con 0x7fc5181013e0 2023-08-11T15:46:24.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.035+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5181a0930 con 0x7fc5181013e0 2023-08-11T15:46:24.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.035+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5181a0df0 con 0x7fc5181013e0 2023-08-11T15:46:24.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.036+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51810c140 con 0x7fc5181013e0 2023-08-11T15:46:24.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.038+0000 7fc5077fe700 1 -- 172.21.15.32:0/3126680830 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7fc50800cd40 con 0x7fc5181013e0 2023-08-11T15:46:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.040+0000 7fc5077fe700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc5000a00a0 0x7fc5000a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.040+0000 7fc5077fe700 1 -- 172.21.15.32:0/3126680830 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc508014070 con 0x7fc5181013e0 2023-08-11T15:46:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.041+0000 7fc515d9b700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc5000a00a0 0x7fc5000a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.041+0000 7fc515d9b700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc5000a00a0 0x7fc5000a2560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000c00 tx=0x7fc50c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:24.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.044+0000 7fc5077fe700 1 -- 172.21.15.32:0/3126680830 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc508051c30 con 0x7fc5181013e0 2023-08-11T15:46:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.338+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc51810c3b0 con 0x7fc5181013e0 2023-08-11T15:46:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.339+0000 7fc5077fe700 1 -- 172.21.15.32:0/3126680830 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fc508095230 con 0x7fc5181013e0 2023-08-11T15:46:24.341 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:24.342 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/4150364023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4150364023},{"type":"v1","addr":"172.21.15.134:6837","nonce":4150364023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:44:52.172791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6841/1060395589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":1060395589},{"type":"v1","addr":"172.21.15.32:6841","nonce":1060395589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:46:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc5000a00a0 msgr2=0x7fc5000a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc5000a00a0 0x7fc5000a2560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000c00 tx=0x7fc50c005fb0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5181013e0 msgr2=0x7fc518198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5181013e0 0x7fc518198800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc50800e9a0 tx=0x7fc50800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 shutdown_connections 2023-08-11T15:46:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fc5000a00a0 0x7fc5000a2560 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5181013e0 0x7fc518198800 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc518106640 0x7fc51819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 --2- 172.21.15.32:0/3126680830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc518103d10 0x7fc518198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 >> 172.21.15.32:0/3126680830 conn(0x7fc5180fad50 msgr2=0x7fc5181028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 shutdown_connections 2023-08-11T15:46:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.341+0000 7fc51cbff700 1 -- 172.21.15.32:0/3126680830 wait complete. 2023-08-11T15:46:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-08-11T15:46:24.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:46:24.425 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:24 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/877816265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:46:24.425 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:24 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2231692968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:46:24.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:24 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/877816265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:46:24.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:24 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2231692968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:46:24.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 -- 172.21.15.32:0/2286477899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 msgr2=0x7ff12c10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2286477899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c10ac70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff12400b0a0 tx=0x7ff12400b3b0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 -- 172.21.15.32:0/2286477899 shutdown_connections 2023-08-11T15:46:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2286477899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c10ac70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2286477899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c074960 0x7ff12c072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2286477899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c073380 0x7ff12c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 -- 172.21.15.32:0/2286477899 >> 172.21.15.32:0/2286477899 conn(0x7ff12c0fbe50 msgr2=0x7ff12c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:24.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 -- 172.21.15.32:0/2286477899 shutdown_connections 2023-08-11T15:46:24.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.564+0000 7ff1331ff700 1 -- 172.21.15.32:0/2286477899 wait complete. 2023-08-11T15:46:24.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.565+0000 7ff1331ff700 1 Processor -- start 2023-08-11T15:46:24.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff1331ff700 1 -- start start 2023-08-11T15:46:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff1331ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c073380 0x7ff12c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff1331ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c074960 0x7ff12c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff1331ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff1331ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1a0460 con 0x7ff12c1087e0 2023-08-11T15:46:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff1331ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1a05a0 con 0x7ff12c074960 2023-08-11T15:46:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff1331ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1a0730 con 0x7ff12c073380 2023-08-11T15:46:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff13179c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff13179c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c19fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51538/0 (socket says 172.21.15.32:51538) 2023-08-11T15:46:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff13179c700 1 -- 172.21.15.32:0/2216387184 learned_addr learned my addr 172.21.15.32:0/2216387184 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff130f9b700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c073380 0x7ff12c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.566+0000 7ff123fff700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c074960 0x7ff12c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff13179c700 1 -- 172.21.15.32:0/2216387184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c073380 msgr2=0x7ff12c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff13179c700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c073380 0x7ff12c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff13179c700 1 -- 172.21.15.32:0/2216387184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c074960 msgr2=0x7ff12c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff13179c700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c074960 0x7ff12c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff13179c700 1 -- 172.21.15.32:0/2216387184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124009d00 con 0x7ff12c1087e0 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff13179c700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c19fe90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff124009fd0 tx=0x7ff12400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff121ffb700 1 -- 172.21.15.32:0/2216387184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400e040 con 0x7ff12c1087e0 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c1a09b0 con 0x7ff12c1087e0 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c1a0e30 con 0x7ff12c1087e0 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff121ffb700 1 -- 172.21.15.32:0/2216387184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff124003e10 con 0x7ff12c1087e0 2023-08-11T15:46:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.567+0000 7ff121ffb700 1 -- 172.21.15.32:0/2216387184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12401b6c0 con 0x7ff12c1087e0 2023-08-11T15:46:24.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.569+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c106e90 con 0x7ff12c1087e0 2023-08-11T15:46:24.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.570+0000 7ff121ffb700 1 -- 172.21.15.32:0/2216387184 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7ff12401d470 con 0x7ff12c1087e0 2023-08-11T15:46:24.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.572+0000 7ff121ffb700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff1140a0020 0x7ff1140a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:24.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.572+0000 7ff121ffb700 1 -- 172.21.15.32:0/2216387184 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff1240cc680 con 0x7ff12c1087e0 2023-08-11T15:46:24.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.572+0000 7ff130f9b700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff1140a0020 0x7ff1140a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:24.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.573+0000 7ff130f9b700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff1140a0020 0x7ff1140a24e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff128005950 tx=0x7ff12800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.576+0000 7ff121ffb700 1 -- 172.21.15.32:0/2216387184 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff124096040 con 0x7ff12c1087e0 2023-08-11T15:46:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:24 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/877816265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:46:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:24 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2231692968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:46:24.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.788+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff12c10e4f0 con 0x7ff12c1087e0 2023-08-11T15:46:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.790+0000 7ff121ffb700 1 -- 172.21.15.32:0/2216387184 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24508 (secure 0 0 0) 0x7ff124099660 con 0x7ff12c1087e0 2023-08-11T15:46:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:46:24.795 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:45:56.189759+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:46:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff1140a0020 msgr2=0x7ff1140a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff1140a0020 0x7ff1140a24e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff128005950 tx=0x7ff12800b410 comp rx=0 tx=0).stop 2023-08-11T15:46:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 msgr2=0x7ff12c19fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c19fe90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff124009fd0 tx=0x7ff12400bce0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 shutdown_connections 2023-08-11T15:46:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff1140a0020 0x7ff1140a24e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff12c1087e0 0x7ff12c19fe90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c073380 0x7ff12c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 --2- 172.21.15.32:0/2216387184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c074960 0x7ff12c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.792+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 >> 172.21.15.32:0/2216387184 conn(0x7ff12c0fbe50 msgr2=0x7ff12c10aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.793+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 shutdown_connections 2023-08-11T15:46:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:24.793+0000 7ff1331ff700 1 -- 172.21.15.32:0/2216387184 wait complete. 2023-08-11T15:46:24.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-11T15:44:19.707016+0000', 'modified': '2023-08-11T15:44:52.172791+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.32:6841/1060395589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6840', 'nonce': 1060395589}, {'type': 'v1', 'addr': '172.21.15.32:6841', 'nonce': 1060395589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/945169036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 945169036}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 945169036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-11T15:46:24.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-11T15:46:24.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T15:46:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 -- 172.21.15.32:0/1474939187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c101390 msgr2=0x7f039c103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 --2- 172.21.15.32:0/1474939187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c101390 0x7f039c103780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0398009990 tx=0x7f0398009ca0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 -- 172.21.15.32:0/1474939187 shutdown_connections 2023-08-11T15:46:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 --2- 172.21.15.32:0/1474939187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c101390 0x7f039c103780 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 --2- 172.21.15.32:0/1474939187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f039c1065f0 0x7f039c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 --2- 172.21.15.32:0/1474939187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f039c103cc0 0x7f039c1060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 -- 172.21.15.32:0/1474939187 >> 172.21.15.32:0/1474939187 conn(0x7f039c0fad80 msgr2=0x7f039c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 -- 172.21.15.32:0/1474939187 shutdown_connections 2023-08-11T15:46:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.020+0000 7f03a47ff700 1 -- 172.21.15.32:0/1474939187 wait complete. 2023-08-11T15:46:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.021+0000 7f03a47ff700 1 Processor -- start 2023-08-11T15:46:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.022+0000 7f03a47ff700 1 -- start start 2023-08-11T15:46:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.022+0000 7f03a47ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f039c101390 0x7f039c100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.022+0000 7f03a47ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c103cc0 0x7f039c0ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.022+0000 7f03a47ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f039c1065f0 0x7f039c0ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.022+0000 7f03a47ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c100020 con 0x7f039c103cc0 2023-08-11T15:46:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.022+0000 7f03a47ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c100190 con 0x7f039c101390 2023-08-11T15:46:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.022+0000 7f03a47ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1a0690 con 0x7f039c1065f0 2023-08-11T15:46:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c103cc0 0x7f039c0ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c103cc0 0x7f039c0ff340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51566/0 (socket says 172.21.15.32:51566) 2023-08-11T15:46:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a259b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f039c101390 0x7f039c100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 -- 172.21.15.32:0/817205954 learned_addr learned my addr 172.21.15.32:0/817205954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:25.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a2d9c700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f039c1065f0 0x7f039c0ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 -- 172.21.15.32:0/817205954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f039c1065f0 msgr2=0x7f039c0ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f039c1065f0 0x7f039c0ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 -- 172.21.15.32:0/817205954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f039c101390 msgr2=0x7f039c100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f039c101390 0x7f039c100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 -- 172.21.15.32:0/817205954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03980095f0 con 0x7f039c103cc0 2023-08-11T15:46:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c103cc0 0x7f039c0ff340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e9a0 tx=0x7f038c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.023+0000 7f03937fe700 1 -- 172.21.15.32:0/817205954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c009790 con 0x7f039c103cc0 2023-08-11T15:46:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.024+0000 7f03937fe700 1 -- 172.21.15.32:0/817205954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f038c010430 con 0x7f039c103cc0 2023-08-11T15:46:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.024+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c1a0970 con 0x7f039c103cc0 2023-08-11T15:46:25.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.024+0000 7f03937fe700 1 -- 172.21.15.32:0/817205954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c018680 con 0x7f039c103cc0 2023-08-11T15:46:25.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.024+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c1a0e80 con 0x7f039c103cc0 2023-08-11T15:46:25.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.027+0000 7f03937fe700 1 -- 172.21.15.32:0/817205954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 134722+0+0 (secure 0 0 0) 0x7f038c0105a0 con 0x7f039c103cc0 2023-08-11T15:46:25.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.028+0000 7f03b81afb80 1 Processor -- start 2023-08-11T15:46:25.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03b81afb80 1 -- start start 2023-08-11T15:46:25.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0aebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aec3e0 0x5564f0aec860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0aed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03b81afb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f0a445e0 con 0x5564f0aec3e0 2023-08-11T15:46:25.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03b81afb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f0a41f00 con 0x5564f0aeba80 2023-08-11T15:46:25.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03b81afb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f0a3d7f0 con 0x5564f0aecda0 2023-08-11T15:46:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03a259b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aec3e0 0x5564f0aec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03937fe700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f038809ffe0 0x7f03880a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03a259b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aec3e0 0x5564f0aec860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51582/0 (socket says 172.21.15.32:51582) 2023-08-11T15:46:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03a259b700 1 -- 172.21.15.32:0/777447335 learned_addr learned my addr 172.21.15.32:0/777447335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03a2d9c700 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0aebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0aed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.029+0000 7f03937fe700 1 -- 172.21.15.32:0/817205954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f038c014070 con 0x7f039c103cc0 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a259b700 1 -- 172.21.15.32:0/777447335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 msgr2=0x5564f0aed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a259b700 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0aed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a259b700 1 -- 172.21.15.32:0/777447335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 msgr2=0x5564f0aebea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a259b700 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0aebea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a259b700 1 -- 172.21.15.32:0/777447335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f08bd5e0 con 0x5564f0aec3e0 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a2d9c700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f038809ffe0 0x7f03880a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a259b700 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aec3e0 0x5564f0aec860 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0398015000 tx=0x7f0398005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef497175c07845b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f03a2d9c700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f038809ffe0 0x7f03880a24a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0394009ec0 tx=0x7f0394004ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f0390ff9700 1 -- 172.21.15.32:0/777447335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039801d070 con 0x5564f0aec3e0 2023-08-11T15:46:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f0390ff9700 1 -- 172.21.15.32:0/777447335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f039801e040 con 0x5564f0aec3e0 2023-08-11T15:46:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.030+0000 7f0390ff9700 1 -- 172.21.15.32:0/777447335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980048f0 con 0x5564f0aec3e0 2023-08-11T15:46:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 -- 172.21.15.32:0/777447335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aec3e0 msgr2=0x5564f0aec860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aec3e0 0x5564f0aec860 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0398015000 tx=0x7f0398005ea0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 -- 172.21.15.32:0/777447335 shutdown_connections 2023-08-11T15:46:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0aed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0aebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 --2- 172.21.15.32:0/777447335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aec3e0 0x5564f0aec860 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 -- 172.21.15.32:0/777447335 >> 172.21.15.32:0/777447335 conn(0x5564f0a18900 msgr2=0x5564f0ae63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 -- 172.21.15.32:0/777447335 shutdown_connections 2023-08-11T15:46:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.031+0000 7f03b81afb80 1 -- 172.21.15.32:0/777447335 wait complete. 2023-08-11T15:46:25.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.032+0000 7f03b81afb80 1 -- start start 2023-08-11T15:46:25.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.032+0000 7f03b81afb80 1 Processor -- start 2023-08-11T15:46:25.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0af5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aec3e0 0x5564f0af5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0afc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03b81afb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f09dba70 con 0x5564f0aecda0 2023-08-11T15:46:25.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03b81afb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f09db770 con 0x5564f0aeba80 2023-08-11T15:46:25.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03b81afb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f08bf8d0 con 0x5564f0aec3e0 2023-08-11T15:46:25.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03b81afb80 10 client.0 _ll_register_callbacks cb 0x5564f09dea60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:46:25.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0af5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0af5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:36068/0 (socket says 172.21.15.32:36068) 2023-08-11T15:46:25.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 -- 172.21.15.32:0/1162077045 learned_addr learned my addr 172.21.15.32:0/1162077045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:25.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aec3e0 0x5564f0af5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a2d9c700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0afc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 -- 172.21.15.32:0/1162077045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aec3e0 msgr2=0x5564f0af5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aec3e0 0x5564f0af5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 -- 172.21.15.32:0/1162077045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 msgr2=0x5564f0afc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0afc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a259b700 1 -- 172.21.15.32:0/1162077045 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c109650 con 0x5564f0aeba80 2023-08-11T15:46:25.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.033+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aec3e0 0x5564f0af5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:25.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f03a259b700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0af5050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0398009ce0 tx=0x7f0398022a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee38335d1e8b365f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:25.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f0385ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:46:25.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f0385ffb700 1 -- 172.21.15.32:0/1162077045 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039801d030 con 0x5564f0aeba80 2023-08-11T15:46:25.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564f0adea40 con 0x5564f0aeba80 2023-08-11T15:46:25.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f0385ffb700 1 -- 172.21.15.32:0/1162077045 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f03980058e0 con 0x5564f0aeba80 2023-08-11T15:46:25.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f0385ffb700 1 -- 172.21.15.32:0/1162077045 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0398005610 con 0x5564f0aeba80 2023-08-11T15:46:25.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f03b81afb80 10 client.25292 fetch_fsmap learned FSMap version 21 2023-08-11T15:46:25.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.034+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5564f0869d90 con 0x5564f0aeba80 2023-08-11T15:46:25.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f0385ffb700 1 -- 172.21.15.32:0/1162077045 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564f0adea40 con 0x5564f0aeba80 2023-08-11T15:46:25.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f0385ffb700 1 -- 172.21.15.32:0/1162077045 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f039802c400 con 0x5564f0aeba80 2023-08-11T15:46:25.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f03b81afb80 10 client.25292 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:46:25.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f03b81afb80 10 client.25292 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:46:25.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f03b81afb80 20 client.25292 populate_metadata read hostname 'smithi032' 2023-08-11T15:46:25.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b00790 0x5564f0aff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f03b81afb80 4 client.25292 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:46:25.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564f0aba490 con 0x5564f0b00790 2023-08-11T15:46:25.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.035+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b00790 0x5564f0aff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.036+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b00790 0x5564f0aff500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b8fa0b576deeed server_cookie=a3b4107cf67f703e in_seq=0 out_seq=0 2023-08-11T15:46:25.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.036+0000 7f0385ffb700 10 client.25292 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:25.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.036+0000 7f0385ffb700 1 -- 172.21.15.32:0/1162077045 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5564f0aba490 con 0x5564f0b00790 2023-08-11T15:46:25.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.036+0000 7f0385ffb700 10 client.25292 handle_command_reply: tid=0 2023-08-11T15:46:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.036+0000 7f03b81afb80 1 client.25292 shutdown 2023-08-11T15:46:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 msgr2=0x5564f0af5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0af5050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0398009ce0 tx=0x7f0398022a60 comp rx=0 tx=0).stop 2023-08-11T15:46:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 shutdown_connections 2023-08-11T15:46:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b00790 0x5564f0aff500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0afc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aeba80 0x5564f0af5050 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f0385ffb700 0 client.25292 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:25.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1162077045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aec3e0 0x5564f0af5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 >> 172.21.15.32:0/1162077045 conn(0x5564f0a18900 msgr2=0x5564f0ae0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:25.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 shutdown_connections 2023-08-11T15:46:25.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 1 -- 172.21.15.32:0/1162077045 wait complete. 2023-08-11T15:46:25.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.037+0000 7f03b81afb80 20 client.25292 trim_cache size 0 max 16384 2023-08-11T15:46:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 Processor -- start 2023-08-11T15:46:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 -- start start 2023-08-11T15:46:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aeba80 0x5564f0ae6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0ae63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f08cbde0 con 0x5564f0aecda0 2023-08-11T15:46:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f09d7d50 con 0x5564f0aec3e0 2023-08-11T15:46:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.055+0000 7f03b81afb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f0a06ef0 con 0x5564f0aeba80 2023-08-11T15:46:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a2d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0ae63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a1d9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a259b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aeba80 0x5564f0ae6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a2d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0ae63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51600/0 (socket says 172.21.15.32:51600) 2023-08-11T15:46:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a2d9c700 1 -- 172.21.15.32:0/1282208123 learned_addr learned my addr 172.21.15.32:0/1282208123 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a1d9a700 1 -- 172.21.15.32:0/1282208123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aeba80 msgr2=0x5564f0ae6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aeba80 0x5564f0ae6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a1d9a700 1 -- 172.21.15.32:0/1282208123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 msgr2=0x5564f0ae63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0ae63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a1d9a700 1 -- 172.21.15.32:0/1282208123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f0aed450 con 0x5564f0aec3e0 2023-08-11T15:46:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a2d9c700 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0ae63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b090f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f038c0c8f10 tx=0x7f038c08e860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7aae5478ac623c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.056+0000 7f03917fa700 1 -- 172.21.15.32:0/1282208123 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c098c50 con 0x5564f0aec3e0 2023-08-11T15:46:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03917fa700 1 -- 172.21.15.32:0/1282208123 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f038c099580 con 0x5564f0aec3e0 2023-08-11T15:46:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03917fa700 1 -- 172.21.15.32:0/1282208123 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c099360 con 0x5564f0aec3e0 2023-08-11T15:46:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 -- 172.21.15.32:0/1282208123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 msgr2=0x5564f0b090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b090f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f038c0c8f10 tx=0x7f038c08e860 comp rx=0 tx=0).stop 2023-08-11T15:46:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 -- 172.21.15.32:0/1282208123 shutdown_connections 2023-08-11T15:46:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aecda0 0x5564f0ae63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aeba80 0x5564f0ae6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 --2- 172.21.15.32:0/1282208123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b090f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 -- 172.21.15.32:0/1282208123 >> 172.21.15.32:0/1282208123 conn(0x5564f0a18900 msgr2=0x5564f0af3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 -- 172.21.15.32:0/1282208123 shutdown_connections 2023-08-11T15:46:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.057+0000 7f03b81afb80 1 -- 172.21.15.32:0/1282208123 wait complete. 2023-08-11T15:46:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 -- start start 2023-08-11T15:46:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 Processor -- start 2023-08-11T15:46:25.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aeba80 0x5564f0b03370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0b088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f0b08df0 con 0x5564f0aeba80 2023-08-11T15:46:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f0b08f60 con 0x5564f0aec3e0 2023-08-11T15:46:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f0b09120 con 0x5564f0aecda0 2023-08-11T15:46:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03b81afb80 10 client.0 _ll_register_callbacks cb 0x7f0394009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:46:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.058+0000 7f03a259b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aeba80 0x5564f0b03370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aeba80 0x5564f0b03370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51614/0 (socket says 172.21.15.32:51614) 2023-08-11T15:46:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 -- 172.21.15.32:0/2887185633 learned_addr learned my addr 172.21.15.32:0/2887185633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:25.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a2d9c700 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0b088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 -- 172.21.15.32:0/2887185633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 msgr2=0x5564f0b088b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0b088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 -- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 msgr2=0x5564f0b038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 -- 172.21.15.32:0/2887185633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0394017440 con 0x5564f0aeba80 2023-08-11T15:46:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03a259b700 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aeba80 0x5564f0b03370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0398000c00 tx=0x7f0398022710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0338fc9517e6ead server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f0385ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:46:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f0385ffb700 1 -- 172.21.15.32:0/2887185633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039801d040 con 0x5564f0aeba80 2023-08-11T15:46:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f039801e040 con 0x5564f0aeba80 2023-08-11T15:46:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.059+0000 7f0385ffb700 1 -- 172.21.15.32:0/2887185633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f039802cb00 con 0x5564f0aeba80 2023-08-11T15:46:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f0385ffb700 1 -- 172.21.15.32:0/2887185633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0398022cd0 con 0x5564f0aeba80 2023-08-11T15:46:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 10 client.15285 fetch_fsmap learned FSMap version 21 2023-08-11T15:46:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7f038c0b6e70 con 0x5564f0aeba80 2023-08-11T15:46:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f0385ffb700 1 -- 172.21.15.32:0/2887185633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f039801e040 con 0x5564f0aeba80 2023-08-11T15:46:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f0385ffb700 1 -- 172.21.15.32:0/2887185633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0398038030 con 0x5564f0aeba80 2023-08-11T15:46:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 10 client.15285 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:46:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 10 client.15285 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T15:46:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 20 client.15285 populate_metadata read hostname 'smithi032' 2023-08-11T15:46:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b58860 0x5564f0b54ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 4 client.15285 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-08-11T15:46:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.060+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5564f0a367d0 con 0x5564f0b58860 2023-08-11T15:46:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b58860 0x5564f0b54ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03a1d9a700 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b58860 0x5564f0b54ce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad5407539a0c979e server_cookie=a6a19e34deaf50c9 in_seq=0 out_seq=0 2023-08-11T15:46:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f0385ffb700 10 client.15285 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f0385ffb700 1 -- 172.21.15.32:0/2887185633 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5564f0a367d0 con 0x5564f0b58860 2023-08-11T15:46:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f0385ffb700 10 client.15285 handle_command_reply: tid=0 2023-08-11T15:46:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 client.15285 shutdown 2023-08-11T15:46:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aeba80 msgr2=0x5564f0b03370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aeba80 0x5564f0b03370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0398000c00 tx=0x7f0398022710 comp rx=0 tx=0).stop 2023-08-11T15:46:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 shutdown_connections 2023-08-11T15:46:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5564f0b58860 0x5564f0b54ce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564f0aeba80 0x5564f0b03370 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5564f0aecda0 0x5564f0b088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 --2- 172.21.15.32:0/2887185633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564f0aec3e0 0x5564f0b038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f0385ffb700 0 client.15285 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.061+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 >> 172.21.15.32:0/2887185633 conn(0x5564f0a18900 msgr2=0x5564f0ae6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 shutdown_connections 2023-08-11T15:46:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03b81afb80 1 -- 172.21.15.32:0/2887185633 wait complete. 2023-08-11T15:46:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03b81afb80 20 client.15285 trim_cache size 0 max 16384 2023-08-11T15:46:25.076 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:46:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "cd36a838-385c-11ee-9b37-001a4aab830c", 2023-08-11T15:46:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-08-11T15:46:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24578, 2023-08-11T15:46:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-08-11T15:46:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-08-11T15:46:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-08-11T15:46:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 124.89813026500001, 2023-08-11T15:46:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 21, 2023-08-11T15:46:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 92, 2023-08-11T15:46:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 88, 2023-08-11T15:46:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 133.09750672000001 2023-08-11T15:46:25.078 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:46:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f038809ffe0 msgr2=0x7f03880a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f038809ffe0 0x7f03880a24a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0394009ec0 tx=0x7f0394004ad0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c103cc0 msgr2=0x7f039c0ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c103cc0 0x7f039c0ff340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e9a0 tx=0x7f038c00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 shutdown_connections 2023-08-11T15:46:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f038809ffe0 0x7f03880a24a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f039c1065f0 0x7f039c0ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f039c101390 0x7f039c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 --2- 172.21.15.32:0/817205954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f039c103cc0 0x7f039c0ff340 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.062+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 >> 172.21.15.32:0/817205954 conn(0x7f039c0fad80 msgr2=0x7f039c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.063+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 shutdown_connections 2023-08-11T15:46:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:25.063+0000 7f03a47ff700 1 -- 172.21.15.32:0/817205954 wait complete. 2023-08-11T15:46:25.082 INFO:teuthology.run_tasks:Running task kclient... 2023-08-11T15:46:25.096 INFO:tasks.kclient:Mounting kernel clients... 2023-08-11T15:46:25.097 INFO:tasks.kclient:config is {'client.0': {}} 2023-08-11T15:46:25.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:25.098 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-08-11T15:46:25.142 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-08-11T15:46:25.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:25.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-11T15:46:25.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:25.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-08-11T15:46:25.333 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:25 smithi032 ceph-mon[97103]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:25.333 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:25 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3126680830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:46:25.334 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:25 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2216387184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:25.369 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-08-11T15:46:25.370 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T15:46:25.370 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:25.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph fs ls 2023-08-11T15:46:25.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:25 smithi134 ceph-mon[130447]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:25.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:25 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3126680830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:46:25.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:25 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2216387184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:25.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:25 smithi192 ceph-mon[130806]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:46:25.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:25 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3126680830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:46:25.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:25 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2216387184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:46:26.109 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:46:26.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:26 smithi134 ceph-mon[130447]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-11T15:46:26.518 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:26 smithi032 ceph-mon[97103]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-11T15:46:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:26 smithi192 ceph-mon[130806]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-11T15:46:27.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:27 smithi134 ceph-mon[130447]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:27.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:27 smithi032 ceph-mon[97103]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:27.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.564+0000 7f3af79b7700 1 -- 172.21.15.32:0/2937674797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0106940 msgr2=0x7f3af0106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:27.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.564+0000 7f3af79b7700 1 --2- 172.21.15.32:0/2937674797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0106940 0x7f3af0106d20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009990 tx=0x7f3aec009ca0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.565+0000 7f3af79b7700 1 -- 172.21.15.32:0/2937674797 shutdown_connections 2023-08-11T15:46:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.565+0000 7f3af79b7700 1 --2- 172.21.15.32:0/2937674797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0106940 0x7f3af0106d20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.565+0000 7f3af79b7700 1 --2- 172.21.15.32:0/2937674797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3af0101300 0x7f3af010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.565+0000 7f3af79b7700 1 --2- 172.21.15.32:0/2937674797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af0100940 0x7f3af0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.565+0000 7f3af79b7700 1 -- 172.21.15.32:0/2937674797 >> 172.21.15.32:0/2937674797 conn(0x7f3af00fc410 msgr2=0x7f3af00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:27.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.565+0000 7f3af79b7700 1 -- 172.21.15.32:0/2937674797 shutdown_connections 2023-08-11T15:46:27.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.565+0000 7f3af79b7700 1 -- 172.21.15.32:0/2937674797 wait complete. 2023-08-11T15:46:27.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 Processor -- start 2023-08-11T15:46:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 -- start start 2023-08-11T15:46:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af0100940 0x7f3af019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101300 0x7f3af019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3af0106940 0x7f3af0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0199100 con 0x7f3af0101300 2023-08-11T15:46:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0199270 con 0x7f3af0100940 2023-08-11T15:46:27.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.566+0000 7f3af79b7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0199430 con 0x7f3af0106940 2023-08-11T15:46:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101300 0x7f3af019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101300 0x7f3af019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51632/0 (socket says 172.21.15.32:51632) 2023-08-11T15:46:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 -- 172.21.15.32:0/4182247053 learned_addr learned my addr 172.21.15.32:0/4182247053 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af5753700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af0100940 0x7f3af019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:27.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af5f54700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3af0106940 0x7f3af0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 -- 172.21.15.32:0/4182247053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3af0106940 msgr2=0x7f3af0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3af0106940 0x7f3af0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 -- 172.21.15.32:0/4182247053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af0100940 msgr2=0x7f3af019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af0100940 0x7f3af019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 -- 172.21.15.32:0/4182247053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0095f0 con 0x7f3af0101300 2023-08-11T15:46:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.567+0000 7f3af4f52700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101300 0x7f3af019ef50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000d950 tx=0x7f3ae000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.568+0000 7f3ae67fc700 1 -- 172.21.15.32:0/4182247053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000f890 con 0x7f3af0101300 2023-08-11T15:46:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.568+0000 7f3ae67fc700 1 -- 172.21.15.32:0/4182247053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ae0010460 con 0x7f3af0101300 2023-08-11T15:46:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.568+0000 7f3ae67fc700 1 -- 172.21.15.32:0/4182247053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000e610 con 0x7f3af0101300 2023-08-11T15:46:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.568+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af0199710 con 0x7f3af0101300 2023-08-11T15:46:27.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.568+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af01a5a00 con 0x7f3af0101300 2023-08-11T15:46:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.569+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af004e620 con 0x7f3af0101300 2023-08-11T15:46:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.571+0000 7f3ae67fc700 1 -- 172.21.15.32:0/4182247053 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 134671+0+0 (secure 0 0 0) 0x7f3ae00108d0 con 0x7f3af0101300 2023-08-11T15:46:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.572+0000 7f3ae67fc700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3adc09ffd0 0x7f3adc0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.572+0000 7f3ae67fc700 1 -- 172.21.15.32:0/4182247053 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ae00cb290 con 0x7f3af0101300 2023-08-11T15:46:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.572+0000 7f3af5753700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3adc09ffd0 0x7f3adc0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.573+0000 7f3af5753700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3adc09ffd0 0x7f3adc0a2490 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b5c0 tx=0x7f3aec005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.576+0000 7f3ae67fc700 1 -- 172.21.15.32:0/4182247053 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3ae0094cf0 con 0x7f3af0101300 2023-08-11T15:46:27.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:27 smithi192 ceph-mon[130806]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:46:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.829+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3af00008d0 con 0x7f3af0101300 2023-08-11T15:46:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.830+0000 7f3ae67fc700 1 -- 172.21.15.32:0/4182247053 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3ae0098310 con 0x7f3af0101300 2023-08-11T15:46:27.832 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T15:46:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3adc09ffd0 msgr2=0x7f3adc0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:27.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3adc09ffd0 0x7f3adc0a2490 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b5c0 tx=0x7f3aec005fd0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101300 msgr2=0x7f3af019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:27.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101300 0x7f3af019ef50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000d950 tx=0x7f3ae000dc60 comp rx=0 tx=0).stop 2023-08-11T15:46:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 shutdown_connections 2023-08-11T15:46:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f3adc09ffd0 0x7f3adc0a2490 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3af0106940 0x7f3af0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3af0100940 0x7f3af019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 --2- 172.21.15.32:0/4182247053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3af0101300 0x7f3af019ef50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 >> 172.21.15.32:0/4182247053 conn(0x7f3af00fc410 msgr2=0x7f3af010bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 shutdown_connections 2023-08-11T15:46:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:27.832+0000 7f3af79b7700 1 -- 172.21.15.32:0/4182247053 wait complete. 2023-08-11T15:46:28.299 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T15:46:28.300 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T15:46:28.300 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-08-11T15:46:28.300 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-11T15:46:28.300 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:28.300 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T15:46:28.301 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T15:46:28.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T15:46:28.301 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-11T15:46:28.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:28.302 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-08-11T15:46:28.353 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T15:46:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T15:46:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T15:46:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:46:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-08-11T15:46:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:46:28.355 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:46:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-08-11T15:46:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-08-11T15:46:28.356 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:46:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-08-11T15:46:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-08-11T15:46:28.357 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T15:46:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:46:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-08-11T15:46:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T15:46:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41546sec preferred_lft 41546sec 2023-08-11T15:46:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-08-11T15:46:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-08-11T15:46:28.358 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T15:46:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-08-11T15:46:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-08-11T15:46:28.358 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-11T15:46:28.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:28.359 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:46:28.359 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:46:28.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-08-11T15:46:28.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-08-11T15:46:28.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-08-11T15:46:28.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-11T15:46:28.360 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:46:28.465 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:28 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4182247053' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T15:46:28.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:28 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4182247053' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T15:46:28.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:28.572 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T15:46:28.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:28 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4182247053' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T15:46:28.642 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-08-11T15:46:28.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:28.653 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-08-11T15:46:28.700 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-08-11T15:46:28.701 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T15:46:28.701 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T15:46:28.701 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T15:46:28.701 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T15:46:28.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:28.702 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:46:28.702 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:46:28.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T15:46:28.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T15:46:28.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T15:46:28.703 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:46:28.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:28.881 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-08-11T15:46:28.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:28.926 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-08-11T15:46:29.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:29.009 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:46:29.010 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:46:29.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-11T15:46:29.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-11T15:46:29.010 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:46:29.194 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-11T15:46:29.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:29.195 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:46:29.195 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:46:29.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-11T15:46:29.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-11T15:46:29.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-11T15:46:29.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-11T15:46:29.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-11T15:46:29.197 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:46:29.404 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:29 smithi032 ceph-mon[97103]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:46:29.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:29.451 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-08-11T15:46:29.451 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-08-11T15:46:29.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-08-11T15:46:29.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-08-11T15:46:29.452 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-08-11T15:46:29.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:29 smithi134 ceph-mon[130447]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:46:29.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:29 smithi192 ceph-mon[130806]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:46:29.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:46:29.629 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:29.672 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-11T15:46:29.673 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T15:46:29.673 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:29.756 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-08-11T15:46:29.756 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-08-11T15:46:29.756 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/4ac296d7-7135-4c0c-ab47-acc92f9cb432 2023-08-11T15:46:29.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:29.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/4ac296d7-7135-4c0c-ab47-acc92f9cb432 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-08-11T15:46:29.903 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-08-11T15:46:29.903 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2023-08-11T15:46:29.904 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-08-11T15:46:30.408 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-08-11T15:46:30.409 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2023-08-11T15:46:30.409 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-08-11T15:46:30.409 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.134:3300,172.21.15.192:3300" 2023-08-11T15:46:30.409 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:3300,172.21.15.134:3300,172.21.15.192:3300:/volumes/_nogroup/sv_1/4ac296d7-7135-4c0c-ab47-acc92f9cb432 2023-08-11T15:46:30.410 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=cd36a838-385c-11ee-9b37-001a4aab830c" will pass to kernel 2023-08-11T15:46:30.415 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-11T15:46:30.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:30.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:30.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:30.490 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-11T15:46:30.578 INFO:teuthology.orchestra.run.smithi032.stdout:client25300 2023-08-11T15:46:30.578 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:46:30.579 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:46:30.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:30.624 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-11T15:46:30.749 INFO:teuthology.orchestra.run.smithi032.stdout:client25300 2023-08-11T15:46:30.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:30.750 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-11T15:46:30.837 INFO:teuthology.orchestra.run.smithi032.stdout:client25300 2023-08-11T15:46:30.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:46:30.838 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cd36a838-385c-11ee-9b37-001a4aab830c.client25300/status') 2023-08-11T15:46:30.948 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.25300 (3)172.21.15.32:0/3353763935 2023-08-11T15:46:30.948 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-08-11T15:46:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-08-11T15:46:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-08-11T15:46:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:71 bytes copied, 5.8413e-05 s, 1.2 MB/s 2023-08-11T15:46:30.951 INFO:teuthology.run_tasks:Running task check-counter... 2023-08-11T15:46:30.964 INFO:teuthology.run_tasks:Running task exec... 2023-08-11T15:46:30.975 INFO:teuthology.task.exec:Executing custom commands... 2023-08-11T15:46:30.976 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:46:30.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-08-11T15:46:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 -- 172.21.15.32:0/3621939596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924101400 msgr2=0x7fd9241037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/3621939596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924101400 0x7fd9241037f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd914009990 tx=0x7fd914009ca0 comp rx=0 tx=0).stop 2023-08-11T15:46:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 -- 172.21.15.32:0/3621939596 shutdown_connections 2023-08-11T15:46:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/3621939596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924101400 0x7fd9241037f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/3621939596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd924106660 0x7fd924108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/3621939596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd924103d30 0x7fd924106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 -- 172.21.15.32:0/3621939596 >> 172.21.15.32:0/3621939596 conn(0x7fd9240fad70 msgr2=0x7fd9240fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 -- 172.21.15.32:0/3621939596 shutdown_connections 2023-08-11T15:46:31.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.173+0000 7fd92a3ff700 1 -- 172.21.15.32:0/3621939596 wait complete. 2023-08-11T15:46:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 Processor -- start 2023-08-11T15:46:31.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 -- start start 2023-08-11T15:46:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd924101400 0x7fd924198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd924103d30 0x7fd924198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924106660 0x7fd924199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419a320 con 0x7fd924106660 2023-08-11T15:46:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419a490 con 0x7fd924103d30 2023-08-11T15:46:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.174+0000 7fd92a3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419a650 con 0x7fd924101400 2023-08-11T15:46:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924106660 0x7fd924199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd923fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd924101400 0x7fd924198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924106660 0x7fd924199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51644/0 (socket says 172.21.15.32:51644) 2023-08-11T15:46:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd923fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd924101400 0x7fd924198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:57450/0 (socket says 172.21.15.32:57450) 2023-08-11T15:46:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 -- 172.21.15.32:0/2947486543 learned_addr learned my addr 172.21.15.32:0/2947486543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd9237fe700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd924103d30 0x7fd924198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:31.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 -- 172.21.15.32:0/2947486543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd924101400 msgr2=0x7fd924198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd924101400 0x7fd924198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 -- 172.21.15.32:0/2947486543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd924103d30 msgr2=0x7fd924198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd924103d30 0x7fd924198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 -- 172.21.15.32:0/2947486543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140095f0 con 0x7fd924106660 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.175+0000 7fd92899c700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924106660 0x7fd924199cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00ef30 tx=0x7fd91c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.176+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c018070 con 0x7fd924106660 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.176+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91c00f040 con 0x7fd924106660 2023-08-11T15:46:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.176+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00cd00 con 0x7fd924106660 2023-08-11T15:46:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.176+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92419a8d0 con 0x7fd924106660 2023-08-11T15:46:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.176+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9241a10f0 con 0x7fd924106660 2023-08-11T15:46:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.177+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92404e700 con 0x7fd924106660 2023-08-11T15:46:31.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.179+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 134671+0+0 (secure 0 0 0) 0x7fd91c003a00 con 0x7fd924106660 2023-08-11T15:46:31.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.181+0000 7fd9217fa700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd90c09ffd0 0x7fd90c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.182+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd91c0cafd0 con 0x7fd924106660 2023-08-11T15:46:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.182+0000 7fd923fff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd90c09ffd0 0x7fd90c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.182+0000 7fd923fff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd90c09ffd0 0x7fd90c0a2490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b2e0 tx=0x7fd914005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:31.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.186+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd91c094a60 con 0x7fd924106660 2023-08-11T15:46:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:31.458+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd9240fea80 con 0x7fd924106660 2023-08-11T15:46:31.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:31 smithi134 ceph-mon[130447]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:46:31.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:31 smithi032 ceph-mon[97103]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:46:31.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:31 smithi192 ceph-mon[130806]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:46:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.229+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7fd91c003cb0 con 0x7fd924106660 2023-08-11T15:46:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.231+0000 7fd9217fa700 1 -- 172.21.15.32:0/2947486543 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd91c0984f0 con 0x7fd924106660 2023-08-11T15:46:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd90c09ffd0 msgr2=0x7fd90c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd90c09ffd0 0x7fd90c0a2490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b2e0 tx=0x7fd914005fb0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924106660 msgr2=0x7fd924199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924106660 0x7fd924199cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00ef30 tx=0x7fd91c00c3a0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 shutdown_connections 2023-08-11T15:46:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7fd90c09ffd0 0x7fd90c0a2490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd924106660 0x7fd924199cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd924101400 0x7fd924198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 --2- 172.21.15.32:0/2947486543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd924103d30 0x7fd924198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 >> 172.21.15.32:0/2947486543 conn(0x7fd9240fad70 msgr2=0x7fd924100320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 shutdown_connections 2023-08-11T15:46:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.234+0000 7fd92a3ff700 1 -- 172.21.15.32:0/2947486543 wait complete. 2023-08-11T15:46:32.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-08-11T15:46:32.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2947486543' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-11T15:46:32.471 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: ignoring --setuser ceph since I am not root 2023-08-11T15:46:32.471 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: ignoring --setgroup ceph since I am not root 2023-08-11T15:46:32.471 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-mgr[131922]: -- 172.21.15.134:0/2094463933 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c54a3f1e0 con 0x555c54a14800 2023-08-11T15:46:32.472 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:32.401+0000 7f3e05454200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:46:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 -- 172.21.15.32:0/1645552574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c100930 msgr2=0x7ff05c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1645552574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c100930 0x7ff05c100d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009990 tx=0x7ff04c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 -- 172.21.15.32:0/1645552574 shutdown_connections 2023-08-11T15:46:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1645552574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c1024c0 0x7ff05c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1645552574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c100930 0x7ff05c100d50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1645552574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff05c101b20 0x7ff05c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 -- 172.21.15.32:0/1645552574 >> 172.21.15.32:0/1645552574 conn(0x7ff05c0fbe60 msgr2=0x7ff05c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.473+0000 7ff0613ff700 1 -- 172.21.15.32:0/1645552574 shutdown_connections 2023-08-11T15:46:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.474+0000 7ff0613ff700 1 -- 172.21.15.32:0/1645552574 wait complete. 2023-08-11T15:46:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.474+0000 7ff0613ff700 1 Processor -- start 2023-08-11T15:46:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff0613ff700 1 -- start start 2023-08-11T15:46:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff0613ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff05c100930 0x7ff05c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff0613ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c101b20 0x7ff05c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff0613ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c1024c0 0x7ff05c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff0613ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c19a3a0 con 0x7ff05c100930 2023-08-11T15:46:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff0613ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c19a510 con 0x7ff05c1024c0 2023-08-11T15:46:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff0613ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c19a6d0 con 0x7ff05c101b20 2023-08-11T15:46:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff05a7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c101b20 0x7ff05c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff05affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff05c100930 0x7ff05c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff05a7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c101b20 0x7ff05c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:57480/0 (socket says 172.21.15.32:57480) 2023-08-11T15:46:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff05a7fc700 1 -- 172.21.15.32:0/1190731828 learned_addr learned my addr 172.21.15.32:0/1190731828 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.475+0000 7ff05b7fe700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c1024c0 0x7ff05c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff05affd700 1 -- 172.21.15.32:0/1190731828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c101b20 msgr2=0x7ff05c198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff05affd700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c101b20 0x7ff05c198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff05affd700 1 -- 172.21.15.32:0/1190731828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c1024c0 msgr2=0x7ff05c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff05affd700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c1024c0 0x7ff05c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff05affd700 1 -- 172.21.15.32:0/1190731828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff054009c40 con 0x7ff05c100930 2023-08-11T15:46:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff05a7fc700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c101b20 0x7ff05c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff05affd700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff05c100930 0x7ff05c198980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098e0 tx=0x7ff04c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff043fff700 1 -- 172.21.15.32:0/1190731828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c01d070 con 0x7ff05c100930 2023-08-11T15:46:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c0095f0 con 0x7ff05c100930 2023-08-11T15:46:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff043fff700 1 -- 172.21.15.32:0/1190731828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04c021410 con 0x7ff05c100930 2023-08-11T15:46:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff043fff700 1 -- 172.21.15.32:0/1190731828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c017650 con 0x7ff05c100930 2023-08-11T15:46:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.476+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c1a0f90 con 0x7ff05c100930 2023-08-11T15:46:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.477+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c04e700 con 0x7ff05c100930 2023-08-11T15:46:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.479+0000 7ff043fff700 1 -- 172.21.15.32:0/1190731828 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7ff04c021580 con 0x7ff05c100930 2023-08-11T15:46:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.482+0000 7ff043fff700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 0x7ff0440a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.482+0000 7ff05a7fc700 1 -- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 msgr2=0x7ff0440a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.482+0000 7ff043fff700 1 -- 172.21.15.32:0/1190731828 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff04c0cc7d0 con 0x7ff05c100930 2023-08-11T15:46:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.482+0000 7ff05a7fc700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 0x7ff0440a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:46:32.485 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2947486543' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-11T15:46:32.485 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: ignoring --setuser ceph since I am not root 2023-08-11T15:46:32.485 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: ignoring --setgroup ceph since I am not root 2023-08-11T15:46:32.485 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-mgr[132890]: -- 172.21.15.192:0/2009757132 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56330922b1e0 con 0x563309200800 2023-08-11T15:46:32.486 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:32.403+0000 7f794c890200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:46:32.486 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:32.482+0000 7f794c890200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:46:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.486+0000 7ff043fff700 1 -- 172.21.15.32:0/1190731828 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff04c096260 con 0x7ff05c100930 2023-08-11T15:46:32.514 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:32 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2947486543' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-11T15:46:32.514 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: ignoring --setuser ceph since I am not root 2023-08-11T15:46:32.515 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: ignoring --setgroup ceph since I am not root 2023-08-11T15:46:32.515 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:32.511+0000 7f3419fa8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T15:46:32.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.683+0000 7ff05a7fc700 1 -- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 msgr2=0x7ff0440a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:32.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.683+0000 7ff05a7fc700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 0x7ff0440a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T15:46:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.698+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff05c10c190 con 0x7ff05c100930 2023-08-11T15:46:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.703+0000 7ff043fff700 1 -- 172.21.15.32:0/1190731828 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff04c099880 con 0x7ff05c100930 2023-08-11T15:46:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 msgr2=0x7ff0440a24e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:46:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 0x7ff0440a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff05c100930 msgr2=0x7ff05c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff05c100930 0x7ff05c198980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098e0 tx=0x7ff04c009580 comp rx=0 tx=0).stop 2023-08-11T15:46:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 shutdown_connections 2023-08-11T15:46:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7ff0440a0020 0x7ff0440a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff05c100930 0x7ff05c198980 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c1024c0 0x7ff05c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 --2- 172.21.15.32:0/1190731828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff05c101b20 0x7ff05c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 >> 172.21.15.32:0/1190731828 conn(0x7ff05c0fbe60 msgr2=0x7ff05c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.705+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 shutdown_connections 2023-08-11T15:46:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.706+0000 7ff0613ff700 1 -- 172.21.15.32:0/1190731828 wait complete. 2023-08-11T15:46:32.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-08-11T15:46:32.790 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:32.621+0000 7f3419fa8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:46:32.790 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:32.742+0000 7f3419fa8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:46:32.833 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:32.574+0000 7f794c890200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:46:32.834 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:32.668+0000 7f794c890200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:46:32.837 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:32.481+0000 7f3e05454200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T15:46:32.837 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:32.572+0000 7f3e05454200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T15:46:32.837 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:32.666+0000 7f3e05454200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:46:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 -- 172.21.15.32:0/1374287572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca4074960 msgr2=0x7f9ca4072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/1374287572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca4074960 0x7f9ca4072db0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9c940098f0 tx=0x7f9c94009c00 comp rx=0 tx=0).stop 2023-08-11T15:46:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 -- 172.21.15.32:0/1374287572 shutdown_connections 2023-08-11T15:46:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/1374287572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca4074960 0x7f9ca4072db0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/1374287572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca41087f0 0x7f9ca410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/1374287572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 0x7f9ca4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 -- 172.21.15.32:0/1374287572 >> 172.21.15.32:0/1374287572 conn(0x7f9ca40fbe60 msgr2=0x7f9ca40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 -- 172.21.15.32:0/1374287572 shutdown_connections 2023-08-11T15:46:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.932+0000 7f9ca9fff700 1 -- 172.21.15.32:0/1374287572 wait complete. 2023-08-11T15:46:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.933+0000 7f9ca9fff700 1 Processor -- start 2023-08-11T15:46:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca9fff700 1 -- start start 2023-08-11T15:46:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca9fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 0x7f9ca41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca9fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca4074960 0x7f9ca4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca9fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca41087f0 0x7f9ca4199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca9fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca419a3c0 con 0x7f9ca41087f0 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca9fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca419a530 con 0x7f9ca4074960 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca9fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca419a6f0 con 0x7f9ca4073380 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca4074960 0x7f9ca4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 0x7f9ca41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca4074960 0x7f9ca4198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:36160/0 (socket says 172.21.15.32:36160) 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 0x7f9ca41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:57512/0 (socket says 172.21.15.32:57512) 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca2ffd700 1 -- 172.21.15.32:0/2284419357 learned_addr learned my addr 172.21.15.32:0/2284419357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.934+0000 7f9ca3fff700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca41087f0 0x7f9ca4199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca37fe700 1 -- 172.21.15.32:0/2284419357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca4074960 msgr2=0x7f9ca4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca37fe700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca4074960 0x7f9ca4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca37fe700 1 -- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca41087f0 msgr2=0x7f9ca4199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca37fe700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca41087f0 0x7f9ca4199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca37fe700 1 -- 172.21.15.32:0/2284419357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94009520 con 0x7f9ca4073380 2023-08-11T15:46:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca2ffd700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca4074960 0x7f9ca4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca37fe700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 0x7f9ca41989a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009230 tx=0x7f9c9400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/2284419357 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9401c070 con 0x7f9ca4073380 2023-08-11T15:46:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/2284419357 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9400f7d0 con 0x7f9ca4073380 2023-08-11T15:46:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca419a970 con 0x7f9ca4073380 2023-08-11T15:46:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.935+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/2284419357 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94003a00 con 0x7f9ca4073380 2023-08-11T15:46:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.936+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca41a1190 con 0x7f9ca4073380 2023-08-11T15:46:32.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.937+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca4106ea0 con 0x7f9ca4073380 2023-08-11T15:46:32.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.938+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/2284419357 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7f9c94020030 con 0x7f9ca4073380 2023-08-11T15:46:32.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.940+0000 7f9ca0ff9700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 0x7f9c8c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:32.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.940+0000 7f9ca2ffd700 1 -- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 msgr2=0x7f9c8c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.940+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/2284419357 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9c940cc3b0 con 0x7f9ca4073380 2023-08-11T15:46:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.940+0000 7f9ca2ffd700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 0x7f9c8c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:46:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:32.943+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/2284419357 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9c94095e40 con 0x7f9ca4073380 2023-08-11T15:46:33.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:32.872+0000 7f3419fa8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T15:46:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.141+0000 7f9ca2ffd700 1 -- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 msgr2=0x7f9c8c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.141+0000 7f9ca2ffd700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 0x7f9c8c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T15:46:33.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.156+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f9ca404e700 con 0x7f9ca4073380 2023-08-11T15:46:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.161+0000 7f9ca0ff9700 1 -- 172.21.15.32:0/2284419357 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9c94099460 con 0x7f9ca4073380 2023-08-11T15:46:33.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 msgr2=0x7f9c8c0a24e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:46:33.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 0x7f9c8c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 msgr2=0x7f9ca41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:33.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 0x7f9ca41989a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009230 tx=0x7f9c9400b790 comp rx=0 tx=0).stop 2023-08-11T15:46:33.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 shutdown_connections 2023-08-11T15:46:33.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f9c8c0a0020 0x7f9c8c0a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca41087f0 0x7f9ca4199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4073380 0x7f9ca41989a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 --2- 172.21.15.32:0/2284419357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca4074960 0x7f9ca4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 >> 172.21.15.32:0/2284419357 conn(0x7f9ca40fbe60 msgr2=0x7f9ca40fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:33.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 shutdown_connections 2023-08-11T15:46:33.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.164+0000 7f9ca9fff700 1 -- 172.21.15.32:0/2284419357 wait complete. 2023-08-11T15:46:33.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-08-11T15:46:33.198 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:32.835+0000 7f3e05454200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:46:33.203 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:32.839+0000 7f794c890200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:46:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.388+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1599940652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19a4103cd0 msgr2=0x7f19a41060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.388+0000 7f19aa1ff700 1 --2- 172.21.15.32:0/1599940652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19a4103cd0 0x7f19a41060c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1998009990 tx=0x7f1998009ca0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.388+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1599940652 shutdown_connections 2023-08-11T15:46:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.388+0000 7f19aa1ff700 1 --2- 172.21.15.32:0/1599940652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a4106600 0x7f19a4108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.388+0000 7f19aa1ff700 1 --2- 172.21.15.32:0/1599940652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19a41013a0 0x7f19a4103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.388+0000 7f19aa1ff700 1 --2- 172.21.15.32:0/1599940652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19a4103cd0 0x7f19a41060c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.388+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1599940652 >> 172.21.15.32:0/1599940652 conn(0x7f19a40fad70 msgr2=0x7f19a40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.389+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1599940652 shutdown_connections 2023-08-11T15:46:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.389+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1599940652 wait complete. 2023-08-11T15:46:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.389+0000 7f19aa1ff700 1 Processor -- start 2023-08-11T15:46:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19aa1ff700 1 -- start start 2023-08-11T15:46:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19aa1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19a41013a0 0x7f19a40ff350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19aa1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a4103cd0 0x7f19a40ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19aa1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19a4106600 0x7f19a40ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19aa1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41a06a0 con 0x7f19a4106600 2023-08-11T15:46:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19aa1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41a07e0 con 0x7f19a41013a0 2023-08-11T15:46:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19aa1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41a0920 con 0x7f19a4103cd0 2023-08-11T15:46:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19a2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a4103cd0 0x7f19a40ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19a37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19a41013a0 0x7f19a40ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19a4106600 0x7f19a40ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19a2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a4103cd0 0x7f19a40ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:57522/0 (socket says 172.21.15.32:57522) 2023-08-11T15:46:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.390+0000 7f19a2ffd700 1 -- 172.21.15.32:0/1150477841 learned_addr learned my addr 172.21.15.32:0/1150477841 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a37fe700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a4103cd0 msgr2=0x7f19a40ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a37fe700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a4103cd0 0x7f19a40ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a37fe700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19a4106600 msgr2=0x7f19a40ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a37fe700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19a4106600 0x7f19a40ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a37fe700 1 -- 172.21.15.32:0/1150477841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19980095f0 con 0x7f19a41013a0 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a4103cd0 0x7f19a40ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a37fe700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19a41013a0 0x7f19a40ff350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f199400e9f0 tx=0x7f199400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199400c9c0 con 0x7f19a41013a0 2023-08-11T15:46:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1994010430 con 0x7f19a41013a0 2023-08-11T15:46:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994018680 con 0x7f19a41013a0 2023-08-11T15:46:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.391+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a41a0bb0 con 0x7f19a41013a0 2023-08-11T15:46:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.392+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a41a1070 con 0x7f19a41013a0 2023-08-11T15:46:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.392+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a40fc8f0 con 0x7f19a41013a0 2023-08-11T15:46:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.395+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 19) v1 ==== 134688+0+0 (secure 0 0 0) 0x7f1994010860 con 0x7f19a41013a0 2023-08-11T15:46:33.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.396+0000 7f19a0ff9700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:33.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.396+0000 7f19a2ffd700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 msgr2=0x7f198c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:33.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.396+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1994014070 con 0x7f19a41013a0 2023-08-11T15:46:33.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.396+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T15:46:33.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.398+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1994095080 con 0x7f19a41013a0 2023-08-11T15:46:33.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2947486543' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-11T15:46:33.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-mon[130447]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-08-11T15:46:33.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1190731828' entity='client.admin' 2023-08-11T15:46:33.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' 2023-08-11T15:46:33.471 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:33.196+0000 7f3e05454200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:46:33.471 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:33.328+0000 7f3e05454200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:46:33.471 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:33.416+0000 7f3e05454200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:46:33.524 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2947486543' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-11T15:46:33.524 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-mon[130806]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-08-11T15:46:33.524 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1190731828' entity='client.admin' 2023-08-11T15:46:33.524 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' 2023-08-11T15:46:33.525 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:33.202+0000 7f794c890200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:46:33.525 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:33.339+0000 7f794c890200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:46:33.525 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:33.428+0000 7f794c890200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:46:33.525 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:33.522+0000 7f794c890200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:46:33.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:33.107+0000 7f3419fa8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T15:46:33.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2947486543' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-11T15:46:33.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-mon[97103]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-08-11T15:46:33.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1190731828' entity='client.admin' 2023-08-11T15:46:33.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' 2023-08-11T15:46:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.597+0000 7f19a2ffd700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 msgr2=0x7f198c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.597+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T15:46:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.639+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 --> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f19a402cce0 con 0x7f198c0a0020 2023-08-11T15:46:33.885 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:33.594+0000 7f3419fa8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T15:46:33.885 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:33.762+0000 7f3419fa8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T15:46:33.886 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:33.882+0000 7f3419fa8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T15:46:33.930 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:33.511+0000 7f3e05454200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:46:34.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.998+0000 7f19a2ffd700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 msgr2=0x7f198c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:34.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:33.998+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-11T15:46:34.220 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:33.928+0000 7f3e05454200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:46:34.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:34.011+0000 7f3419fa8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T15:46:34.333 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:33.942+0000 7f794c890200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:46:34.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:34.799+0000 7f19a2ffd700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 msgr2=0x7f198c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:34.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:34.799+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-11T15:46:34.970 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:34.584+0000 7f3e05454200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:46:34.971 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:34.666+0000 7f3e05454200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:46:35.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:34.573+0000 7f3419fa8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T15:46:35.083 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:34.598+0000 7f794c890200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:46:35.083 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:34.682+0000 7f794c890200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:46:35.698 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:35.435+0000 7f3e05454200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:46:35.698 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:35.525+0000 7f3e05454200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:46:35.719 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:35.455+0000 7f794c890200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:46:35.719 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:35.546+0000 7f794c890200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:46:35.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:35 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:35.456+0000 7f3419fa8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T15:46:35.803 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:35 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:35.567+0000 7f3419fa8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T15:46:35.970 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:35.696+0000 7f3e05454200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:46:36.001 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:35.717+0000 7f794c890200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:46:36.334 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:35.999+0000 7f794c890200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:46:36.334 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:36.227+0000 7f794c890200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:46:36.354 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:35.976+0000 7f3e05454200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:46:36.355 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:36.205+0000 7f3e05454200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:46:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:36.400+0000 7f19a2ffd700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 msgr2=0x7f198c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:36.400+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-11T15:46:36.616 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:36.352+0000 7f3e05454200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:46:36.638 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:36.375+0000 7f794c890200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:46:36.925 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:36.563+0000 7f3419fa8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:46:36.926 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:36.687+0000 7f3419fa8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T15:46:36.970 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:36.615+0000 7f3e05454200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:46:37.083 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:36.637+0000 7f794c890200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:46:37.302 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:36.923+0000 7f3419fa8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T15:46:37.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:37 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:37.299+0000 7f3419fa8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T15:46:37.637 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:37.313+0000 7f3e05454200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:46:37.637 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:37.443+0000 7f3e05454200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:46:37.664 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:37.339+0000 7f794c890200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:46:37.664 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:37.470+0000 7f794c890200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:46:37.966 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:37.635+0000 7f3e05454200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:46:37.966 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:37.748+0000 7f3e05454200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:46:37.994 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:37.662+0000 7f794c890200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:46:37.994 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:37.776+0000 7f794c890200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:46:38.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:37 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:37.583+0000 7f3419fa8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T15:46:38.053 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:37 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:37.786+0000 7f3419fa8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T15:46:38.220 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:37.964+0000 7f3e05454200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:46:38.333 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:37.992+0000 7f794c890200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:46:38.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:38.149+0000 7f3419fa8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T15:46:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:38.899+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mgrmap(e 20) v1 ==== 134688+0+0 (secure 0 0 0) 0x7f1994095450 con 0x7f19a41013a0 2023-08-11T15:46:39.220 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y[131897]: 2023-08-11T15:46:38.822+0000 7f3e05454200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:46:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: Standby manager daemon y restarted 2023-08-11T15:46:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: Standby manager daemon y started 2023-08-11T15:46:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T15:46:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:46:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T15:46:39.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:46:39.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: Standby manager daemon z restarted 2023-08-11T15:46:39.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: Standby manager daemon z started 2023-08-11T15:46:39.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T15:46:39.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:46:39.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T15:46:39.223 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:38 smithi134 ceph-mon[130447]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:46:39.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:39.121+0000 7f3419fa8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T15:46:39.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: Standby manager daemon y restarted 2023-08-11T15:46:39.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: Standby manager daemon y started 2023-08-11T15:46:39.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T15:46:39.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:46:39.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T15:46:39.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:46:39.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: Standby manager daemon z restarted 2023-08-11T15:46:39.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: Standby manager daemon z started 2023-08-11T15:46:39.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T15:46:39.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:46:39.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T15:46:39.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:38 smithi032 ceph-mon[97103]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:46:39.333 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z[132804]: 2023-08-11T15:46:38.852+0000 7f794c890200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:46:39.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: Standby manager daemon y restarted 2023-08-11T15:46:39.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: Standby manager daemon y started 2023-08-11T15:46:39.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T15:46:39.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:46:39.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T15:46:39.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.134:0/878601798' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:46:39.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: Standby manager daemon z restarted 2023-08-11T15:46:39.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: Standby manager daemon z started 2023-08-11T15:46:39.338 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T15:46:39.338 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T15:46:39.339 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T15:46:39.339 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:38 smithi192 ceph-mon[130806]: from='mgr.? 172.21.15.192:0/3704396388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T15:46:39.570 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:39.304+0000 7f3419fa8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T15:46:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:39.603+0000 7f19a2ffd700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 msgr2=0x7f198c0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2482310260 2023-08-11T15:46:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:39.603+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-11T15:46:39.904 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:39.567+0000 7f3419fa8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T15:46:39.904 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:39.722+0000 7f3419fa8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T15:46:40.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:39 smithi134 ceph-mon[130447]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-08-11T15:46:40.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:39 smithi032 ceph-mon[97103]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-08-11T15:46:40.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:40.012+0000 7f3419fa8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T15:46:40.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:39 smithi192 ceph-mon[130806]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-08-11T15:46:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:41.149+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 8 ==== mgrmap(e 21) v1 ==== 134356+0+0 (secure 0 0 0) 0x7f19940c8e70 con 0x7f19a41013a0 2023-08-11T15:46:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:41.150+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 msgr2=0x7f198c0a24e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T15:46:41.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:41.150+0000 7f19a0ff9700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/2482310260,v1:172.21.15.32:6801/2482310260] conn(0x7f198c0a0020 0x7f198c0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:41.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: Active manager daemon x restarted 2023-08-11T15:46:41.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: Activating manager daemon x 2023-08-11T15:46:41.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: osdmap e93: 12 total, 12 up, 12 in 2023-08-11T15:46:41.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: mgrmap e21: x(active, starting, since 0.0113139s), standbys: z, y 2023-08-11T15:46:41.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T15:46:41.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T15:46:41.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T15:46:41.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T15:46:41.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T15:46:41.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:46:41.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:46:41.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:46:41.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:46:41.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T15:46:41.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T15:46:41.448 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T15:46:41.448 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:41 smithi192 ceph-mon[130806]: Manager daemon x is now available 2023-08-11T15:46:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: Active manager daemon x restarted 2023-08-11T15:46:41.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: Activating manager daemon x 2023-08-11T15:46:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: osdmap e93: 12 total, 12 up, 12 in 2023-08-11T15:46:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: mgrmap e21: x(active, starting, since 0.0113139s), standbys: z, y 2023-08-11T15:46:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:46:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:46:41.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:46:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T15:46:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T15:46:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T15:46:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T15:46:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T15:46:41.473 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T15:46:41.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T15:46:41.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T15:46:41.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T15:46:41.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:46:41.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:46:41.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:46:41.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T15:46:41.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T15:46:41.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T15:46:41.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:46:41.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:46:41.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:46:41.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:46:41.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:46:41.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:46:41.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:46:41.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:46:41.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:46:41.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:46:41.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:46:41.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:46:41.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T15:46:41.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T15:46:41.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T15:46:41.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:41 smithi134 ceph-mon[130447]: Manager daemon x is now available 2023-08-11T15:46:41.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.130+0000 7f3419fa8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T15:46:41.554 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337a15f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.554 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337a15f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.554 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337a15f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.554 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337a15f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.554 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337a15f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.554 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337c964700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.555 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337c964700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.555 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337c964700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.555 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337c964700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.555 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:46:41.367+0000 7f337c964700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:46:41.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: Active manager daemon x restarted 2023-08-11T15:46:41.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: Activating manager daemon x 2023-08-11T15:46:41.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: osdmap e93: 12 total, 12 up, 12 in 2023-08-11T15:46:41.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: mgrmap e21: x(active, starting, since 0.0113139s), standbys: z, y 2023-08-11T15:46:41.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T15:46:41.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T15:46:41.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T15:46:41.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T15:46:41.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T15:46:41.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T15:46:41.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T15:46:41.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T15:46:41.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T15:46:41.558 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T15:46:41.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T15:46:41.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T15:46:41.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T15:46:41.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T15:46:41.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T15:46:41.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T15:46:41.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T15:46:41.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T15:46:41.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T15:46:41.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:41 smithi032 ceph-mon[97103]: Manager daemon x is now available 2023-08-11T15:46:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.152+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mon.2 v2:172.21.15.134:3300/0 9 ==== mgrmap(e 22) v1 ==== 134483+0+0 (secure 0 0 0) 0x7f1994077850 con 0x7f19a41013a0 2023-08-11T15:46:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.153+0000 7f19a0ff9700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f198c01b0d0 0x7f198c01b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.153+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 --> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f19a402cce0 con 0x7f198c01b0d0 2023-08-11T15:46:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.155+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f198c01b0d0 0x7f198c01b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.158+0000 7f19a2ffd700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f198c01b0d0 0x7f198c01b4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1998004800 tx=0x7f1998005fb0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.225+0000 7f19a0ff9700 1 -- 172.21.15.32:0/1150477841 <== mgr.15315 v2:172.21.15.32:6800/1148710426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f19a402cce0 con 0x7f198c01b0d0 2023-08-11T15:46:42.228 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path / 2023-08-11T15:46:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.227+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f198c01b0d0 msgr2=0x7f198c01b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19aa1ff700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f198c01b0d0 0x7f198c01b4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1998004800 tx=0x7f1998005fb0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19a41013a0 msgr2=0x7f19a40ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19aa1ff700 1 --2- 172.21.15.32:0/1150477841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19a41013a0 0x7f19a40ff350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f199400e9f0 tx=0x7f199400ed00 comp rx=0 tx=0).stop 2023-08-11T15:46:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19a3fff700 1 -- 172.21.15.32:0/1150477841 reap_dead start 2023-08-11T15:46:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 shutdown_connections 2023-08-11T15:46:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 >> 172.21.15.32:0/1150477841 conn(0x7f19a40fad70 msgr2=0x7f19a4102010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 shutdown_connections 2023-08-11T15:46:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.228+0000 7f19aa1ff700 1 -- 172.21.15.32:0/1150477841 wait complete. 2023-08-11T15:46:42.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-08-11T15:46:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.458+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2086286874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d4101b20 msgr2=0x7fb7d4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.458+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2086286874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d4101b20 0x7fb7d4101f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80099c0 tx=0x7fb7c8009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.459+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2086286874 shutdown_connections 2023-08-11T15:46:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.459+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2086286874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7d41024c0 0x7fb7d4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.459+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2086286874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 0x7fb7d4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.459+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2086286874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d4101b20 0x7fb7d4101f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.459+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2086286874 >> 172.21.15.32:0/2086286874 conn(0x7fb7d40fbe60 msgr2=0x7fb7d40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.459+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2086286874 shutdown_connections 2023-08-11T15:46:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.459+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2086286874 wait complete. 2023-08-11T15:46:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.460+0000 7fb7da5ff700 1 Processor -- start 2023-08-11T15:46:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.460+0000 7fb7da5ff700 1 -- start start 2023-08-11T15:46:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7da5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 0x7fb7d4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7da5ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7d4101b20 0x7fb7d4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7da5ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d41024c0 0x7fb7d4199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7da5ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d419a370 con 0x7fb7d41024c0 2023-08-11T15:46:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7da5ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d419a4e0 con 0x7fb7d4101b20 2023-08-11T15:46:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7da5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d419a6a0 con 0x7fb7d4100930 2023-08-11T15:46:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 0x7fb7d4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d8b9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d41024c0 0x7fb7d4199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7d4101b20 0x7fb7d4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 0x7fb7d4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:59042/0 (socket says 172.21.15.32:59042) 2023-08-11T15:46:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 -- 172.21.15.32:0/2381190106 learned_addr learned my addr 172.21.15.32:0/2381190106 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 -- 172.21.15.32:0/2381190106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7d4101b20 msgr2=0x7fb7d4198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7d4101b20 0x7fb7d4198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 -- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d41024c0 msgr2=0x7fb7d4199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d41024c0 0x7fb7d4199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.461+0000 7fb7d3fff700 1 -- 172.21.15.32:0/2381190106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80095f0 con 0x7fb7d4100930 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.462+0000 7fb7d3fff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 0x7fb7d4198950 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400e970 tx=0x7fb7c400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.462+0000 7fb7d17fa700 1 -- 172.21.15.32:0/2381190106 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4009790 con 0x7fb7d4100930 2023-08-11T15:46:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.462+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d419a980 con 0x7fb7d4100930 2023-08-11T15:46:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.462+0000 7fb7d17fa700 1 -- 172.21.15.32:0/2381190106 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c400c7e0 con 0x7fb7d4100930 2023-08-11T15:46:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.462+0000 7fb7d17fa700 1 -- 172.21.15.32:0/2381190106 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4003c70 con 0x7fb7d4100930 2023-08-11T15:46:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.462+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d41a1140 con 0x7fb7d4100930 2023-08-11T15:46:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.463+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d404e700 con 0x7fb7d4100930 2023-08-11T15:46:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.465+0000 7fb7d17fa700 1 -- 172.21.15.32:0/2381190106 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 22) v1 ==== 134483+0+0 (secure 0 0 0) 0x7fb7c400cc50 con 0x7fb7d4100930 2023-08-11T15:46:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.466+0000 7fb7d17fa700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb7bc09fd50 0x7fb7bc0a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.466+0000 7fb7d17fa700 1 -- 172.21.15.32:0/2381190106 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb7c4014070 con 0x7fb7d4100930 2023-08-11T15:46:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.467+0000 7fb7d37fe700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb7bc09fd50 0x7fb7bc0a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.467+0000 7fb7d37fe700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb7bc09fd50 0x7fb7bc0a2210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b580 tx=0x7fb7c8005fb0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:42 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:42.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:42 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:42 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T15:46:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:42 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T15:46:42.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:42 smithi134 ceph-mon[130447]: mgrmap e22: x(active, since 1.01738s), standbys: z, y 2023-08-11T15:46:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.469+0000 7fb7d17fa700 1 -- 172.21.15.32:0/2381190106 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb7c40952c0 con 0x7fb7d4100930 2023-08-11T15:46:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:42 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:42 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:42 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T15:46:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:42 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T15:46:42.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:42 smithi032 ceph-mon[97103]: mgrmap e22: x(active, since 1.01738s), standbys: z, y 2023-08-11T15:46:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:42 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:42 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:42 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T15:46:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:42 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T15:46:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:42 smithi192 ceph-mon[130806]: mgrmap e22: x(active, since 1.01738s), standbys: z, y 2023-08-11T15:46:42.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.738+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 --> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7d4108480 con 0x7fb7bc09fd50 2023-08-11T15:46:42.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.798+0000 7fb7d17fa700 1 -- 172.21.15.32:0/2381190106 <== mgr.15315 v2:172.21.15.32:6800/1148710426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fb7d4108480 con 0x7fb7bc09fd50 2023-08-11T15:46:42.801 INFO:teuthology.orchestra.run.smithi032.stdout:Retention added to path / 2023-08-11T15:46:42.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb7bc09fd50 msgr2=0x7fb7bc0a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:42.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb7bc09fd50 0x7fb7bc0a2210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b580 tx=0x7fb7c8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 msgr2=0x7fb7d4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:42.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 0x7fb7d4198950 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400e970 tx=0x7fb7c400ec80 comp rx=0 tx=0).stop 2023-08-11T15:46:42.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 shutdown_connections 2023-08-11T15:46:42.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb7bc09fd50 0x7fb7bc0a2210 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7d41024c0 0x7fb7d4199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d4100930 0x7fb7d4198950 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 --2- 172.21.15.32:0/2381190106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7d4101b20 0x7fb7d4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:42.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 >> 172.21.15.32:0/2381190106 conn(0x7fb7d40fbe60 msgr2=0x7fb7d40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:42.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 shutdown_connections 2023-08-11T15:46:42.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:42.801+0000 7fb7da5ff700 1 -- 172.21.15.32:0/2381190106 wait complete. 2023-08-11T15:46:42.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-08-11T15:46:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.025+0000 7f82287ff700 1 -- 172.21.15.32:0/621478275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82201024c0 msgr2=0x7f8220108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.025+0000 7f82287ff700 1 --2- 172.21.15.32:0/621478275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82201024c0 0x7f8220108d70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f821800b0d0 tx=0x7f821800b3e0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.026+0000 7f82287ff700 1 -- 172.21.15.32:0/621478275 shutdown_connections 2023-08-11T15:46:43.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.026+0000 7f82287ff700 1 --2- 172.21.15.32:0/621478275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82201024c0 0x7f8220108d70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.026+0000 7f82287ff700 1 --2- 172.21.15.32:0/621478275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8220100930 0x7f8220100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.026+0000 7f82287ff700 1 --2- 172.21.15.32:0/621478275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220101b20 0x7f8220101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.026+0000 7f82287ff700 1 -- 172.21.15.32:0/621478275 >> 172.21.15.32:0/621478275 conn(0x7f82200fbe60 msgr2=0x7f82200fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:43.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.026+0000 7f82287ff700 1 -- 172.21.15.32:0/621478275 shutdown_connections 2023-08-11T15:46:43.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.026+0000 7f82287ff700 1 -- 172.21.15.32:0/621478275 wait complete. 2023-08-11T15:46:43.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 Processor -- start 2023-08-11T15:46:43.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 -- start start 2023-08-11T15:46:43.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220100930 0x7f8220198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8220101b20 0x7f8220198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82201024c0 0x7f8220199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019a450 con 0x7f8220101b20 2023-08-11T15:46:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019a5c0 con 0x7f8220100930 2023-08-11T15:46:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.027+0000 7f82287ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019a780 con 0x7f82201024c0 2023-08-11T15:46:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f8225d9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8220101b20 0x7f8220198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f8226d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82201024c0 0x7f8220199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f822659b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220100930 0x7f8220198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f8225d9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8220101b20 0x7f8220198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60168/0 (socket says 172.21.15.32:60168) 2023-08-11T15:46:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f8225d9a700 1 -- 172.21.15.32:0/1899467823 learned_addr learned my addr 172.21.15.32:0/1899467823 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f822659b700 1 -- 172.21.15.32:0/1899467823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82201024c0 msgr2=0x7f8220199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f822659b700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82201024c0 0x7f8220199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f822659b700 1 -- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8220101b20 msgr2=0x7f8220198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f822659b700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8220101b20 0x7f8220198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f822659b700 1 -- 172.21.15.32:0/1899467823 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8218009d00 con 0x7f8220100930 2023-08-11T15:46:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f8225d9a700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8220101b20 0x7f8220198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.028+0000 7f822659b700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220100930 0x7f8220198a30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9c0 tx=0x7f821c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.029+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c004220 con 0x7f8220100930 2023-08-11T15:46:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.029+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821c004380 con 0x7f8220100930 2023-08-11T15:46:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.029+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822019aa60 con 0x7f8220100930 2023-08-11T15:46:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.029+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c003be0 con 0x7f8220100930 2023-08-11T15:46:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.029+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82201a1220 con 0x7f8220100930 2023-08-11T15:46:43.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.030+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822004e700 con 0x7f8220100930 2023-08-11T15:46:43.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.032+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 22) v1 ==== 134483+0+0 (secure 0 0 0) 0x7f821c003d40 con 0x7f8220100930 2023-08-11T15:46:43.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.033+0000 7f82177fe700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f820c09fd50 0x7f820c0a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.033+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f821c014070 con 0x7f8220100930 2023-08-11T15:46:43.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.033+0000 7f8225d9a700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f820c09fd50 0x7f820c0a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.034+0000 7f8225d9a700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f820c09fd50 0x7f820c0a2210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f82100104e0 tx=0x7f8210007480 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:43.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.035+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f821c095390 con 0x7f8220100930 2023-08-11T15:46:43.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.205+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mgrmap(e 23) v1 ==== 134791+0+0 (secure 0 0 0) 0x7f821c095760 con 0x7f8220100930 2023-08-11T15:46:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.307+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 --> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f8220108ac0 con 0x7f820c09fd50 2023-08-11T15:46:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.316+0000 7f82177fe700 1 -- 172.21.15.32:0/1899467823 <== mgr.15315 v2:172.21.15.32:6800/1148710426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f8220108ac0 con 0x7f820c09fd50 2023-08-11T15:46:43.318 INFO:teuthology.orchestra.run.smithi032.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-08-11T00:00:00", "created": "2023-08-11T15:46:42", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-08-11T15:46:43.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f820c09fd50 msgr2=0x7f820c0a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f820c09fd50 0x7f820c0a2210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f82100104e0 tx=0x7f8210007480 comp rx=0 tx=0).stop 2023-08-11T15:46:43.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220100930 msgr2=0x7f8220198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220100930 0x7f8220198a30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9c0 tx=0x7f821c00ecd0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 shutdown_connections 2023-08-11T15:46:43.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f820c09fd50 0x7f820c0a2210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82201024c0 0x7f8220199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8220100930 0x7f8220198a30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 --2- 172.21.15.32:0/1899467823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8220101b20 0x7f8220198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 >> 172.21.15.32:0/1899467823 conn(0x7f82200fbe60 msgr2=0x7f82201073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 shutdown_connections 2023-08-11T15:46:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.318+0000 7f82287ff700 1 -- 172.21.15.32:0/1899467823 wait complete. 2023-08-11T15:46:43.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-08-11T15:46:43.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:43 smithi134 ceph-mon[130447]: [11/Aug/2023:15:46:41] ENGINE Bus STARTING 2023-08-11T15:46:43.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:43 smithi134 ceph-mon[130447]: [11/Aug/2023:15:46:41] ENGINE Serving on http://172.21.15.32:8765 2023-08-11T15:46:43.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:43 smithi134 ceph-mon[130447]: [11/Aug/2023:15:46:41] ENGINE Serving on https://172.21.15.32:7150 2023-08-11T15:46:43.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:43 smithi134 ceph-mon[130447]: [11/Aug/2023:15:46:41] ENGINE Bus STARTED 2023-08-11T15:46:43.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:43 smithi134 ceph-mon[130447]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:43.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:43 smithi134 ceph-mon[130447]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:43.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:43 smithi032 ceph-mon[97103]: [11/Aug/2023:15:46:41] ENGINE Bus STARTING 2023-08-11T15:46:43.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:43 smithi032 ceph-mon[97103]: [11/Aug/2023:15:46:41] ENGINE Serving on http://172.21.15.32:8765 2023-08-11T15:46:43.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:43 smithi032 ceph-mon[97103]: [11/Aug/2023:15:46:41] ENGINE Serving on https://172.21.15.32:7150 2023-08-11T15:46:43.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:43 smithi032 ceph-mon[97103]: [11/Aug/2023:15:46:41] ENGINE Bus STARTED 2023-08-11T15:46:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:43 smithi032 ceph-mon[97103]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:43.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:43 smithi032 ceph-mon[97103]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 -- 172.21.15.32:0/1637834912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874101b40 msgr2=0x7fc874101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 --2- 172.21.15.32:0/1637834912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874101b40 0x7fc874101fa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099c0 tx=0x7fc868009cd0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 -- 172.21.15.32:0/1637834912 shutdown_connections 2023-08-11T15:46:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 --2- 172.21.15.32:0/1637834912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874100950 0x7fc874100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 --2- 172.21.15.32:0/1637834912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8741024e0 0x7fc874108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 --2- 172.21.15.32:0/1637834912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874101b40 0x7fc874101fa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 -- 172.21.15.32:0/1637834912 >> 172.21.15.32:0/1637834912 conn(0x7fc8740fbea0 msgr2=0x7fc8740fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 -- 172.21.15.32:0/1637834912 shutdown_connections 2023-08-11T15:46:43.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.554+0000 7fc879fff700 1 -- 172.21.15.32:0/1637834912 wait complete. 2023-08-11T15:46:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.555+0000 7fc879fff700 1 Processor -- start 2023-08-11T15:46:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.555+0000 7fc879fff700 1 -- start start 2023-08-11T15:46:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc879fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874100950 0x7fc874072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc879fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc874101b40 0x7fc874073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc879fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8741024e0 0x7fc874073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc879fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a0660 con 0x7fc8741024e0 2023-08-11T15:46:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc879fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a07a0 con 0x7fc874100950 2023-08-11T15:46:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc879fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a08e0 con 0x7fc874101b40 2023-08-11T15:46:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc872ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc874101b40 0x7fc874073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc873fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8741024e0 0x7fc874073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc872ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc874101b40 0x7fc874073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:59120/0 (socket says 172.21.15.32:59120) 2023-08-11T15:46:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc872ffd700 1 -- 172.21.15.32:0/2873137290 learned_addr learned my addr 172.21.15.32:0/2873137290 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.556+0000 7fc8737fe700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874100950 0x7fc874072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc872ffd700 1 -- 172.21.15.32:0/2873137290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874100950 msgr2=0x7fc874072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc872ffd700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874100950 0x7fc874072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc872ffd700 1 -- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8741024e0 msgr2=0x7fc874073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc872ffd700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8741024e0 0x7fc874073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc872ffd700 1 -- 172.21.15.32:0/2873137290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009520 con 0x7fc874101b40 2023-08-11T15:46:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc873fff700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8741024e0 0x7fc874073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc872ffd700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc874101b40 0x7fc874073250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc868009fd0 tx=0x7fc868009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc870ff9700 1 -- 172.21.15.32:0/2873137290 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86801d070 con 0x7fc874101b40 2023-08-11T15:46:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680095f0 con 0x7fc874101b40 2023-08-11T15:46:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc870ff9700 1 -- 172.21.15.32:0/2873137290 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc868005030 con 0x7fc874101b40 2023-08-11T15:46:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc870ff9700 1 -- 172.21.15.32:0/2873137290 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868004a90 con 0x7fc874101b40 2023-08-11T15:46:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.557+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741a0e00 con 0x7fc874101b40 2023-08-11T15:46:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.558+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87404e700 con 0x7fc874101b40 2023-08-11T15:46:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.560+0000 7fc870ff9700 1 -- 172.21.15.32:0/2873137290 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) v1 ==== 134791+0+0 (secure 0 0 0) 0x7fc8680051a0 con 0x7fc874101b40 2023-08-11T15:46:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.562+0000 7fc870ff9700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc85c0a00c0 0x7fc85c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.562+0000 7fc870ff9700 1 -- 172.21.15.32:0/2873137290 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc8680cc840 con 0x7fc874101b40 2023-08-11T15:46:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.562+0000 7fc8737fe700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc85c0a00c0 0x7fc85c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.563+0000 7fc8737fe700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc85c0a00c0 0x7fc85c0a2580 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc874074620 tx=0x7fc864009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.564+0000 7fc870ff9700 1 -- 172.21.15.32:0/2873137290 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc868095fb0 con 0x7fc874101b40 2023-08-11T15:46:43.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:43 smithi192 ceph-mon[130806]: [11/Aug/2023:15:46:41] ENGINE Bus STARTING 2023-08-11T15:46:43.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:43 smithi192 ceph-mon[130806]: [11/Aug/2023:15:46:41] ENGINE Serving on http://172.21.15.32:8765 2023-08-11T15:46:43.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:43 smithi192 ceph-mon[130806]: [11/Aug/2023:15:46:41] ENGINE Serving on https://172.21.15.32:7150 2023-08-11T15:46:43.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:43 smithi192 ceph-mon[130806]: [11/Aug/2023:15:46:41] ENGINE Bus STARTED 2023-08-11T15:46:43.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:43 smithi192 ceph-mon[130806]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:43.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:43 smithi192 ceph-mon[130806]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.812+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 --> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc874060d60 con 0x7fc85c0a00c0 2023-08-11T15:46:43.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.820+0000 7fc870ff9700 1 -- 172.21.15.32:0/2873137290 <== mgr.15315 v2:172.21.15.32:6800/1148710426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fc874060d60 con 0x7fc85c0a00c0 2023-08-11T15:46:43.823 INFO:teuthology.orchestra.run.smithi032.stdout:/ 1M 6M3h 2023-08-11T15:46:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.822+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc85c0a00c0 msgr2=0x7fc85c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.822+0000 7fc879fff700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc85c0a00c0 0x7fc85c0a2580 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc874074620 tx=0x7fc864009260 comp rx=0 tx=0).stop 2023-08-11T15:46:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.822+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc874101b40 msgr2=0x7fc874073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.822+0000 7fc879fff700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc874101b40 0x7fc874073250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc868009fd0 tx=0x7fc868009580 comp rx=0 tx=0).stop 2023-08-11T15:46:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 shutdown_connections 2023-08-11T15:46:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc85c0a00c0 0x7fc85c0a2580 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8741024e0 0x7fc874073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc874100950 0x7fc874072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 --2- 172.21.15.32:0/2873137290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc874101b40 0x7fc874073250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 >> 172.21.15.32:0/2873137290 conn(0x7fc8740fbea0 msgr2=0x7fc8740fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 shutdown_connections 2023-08-11T15:46:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:46:43.823+0000 7fc879fff700 1 -- 172.21.15.32:0/2873137290 wait complete. 2023-08-11T15:46:43.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-08-11T15:46:43.935 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-08-11T15:46:43.954 INFO:teuthology.run_tasks:Running task background_exec... 2023-08-11T15:46:43.967 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:46:43.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-08-11T15:46:43.972 INFO:teuthology.run_tasks:Running task workunit... 2023-08-11T15:46:43.994 INFO:tasks.workunit:Pulling workunits from ref 92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:46:43.995 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-08-11T15:46:43.995 INFO:tasks.workunit:timeout=3h 2023-08-11T15:46:43.995 INFO:tasks.workunit:cleanup=True 2023-08-11T15:46:43.996 DEBUG:teuthology.orchestra.run.smithi032:> stat -- /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:44.045 INFO:teuthology.orchestra.run.smithi032.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:44.046 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-08-11T15:46:44.046 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-08-11T15:46:44.046 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T15:46:44.046 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:cephfs_t:s0 2023-08-11T15:46:44.046 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-08-11 15:44:53.723625099 +0000 2023-08-11T15:46:44.047 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-08-11 15:44:53.723625099 +0000 2023-08-11T15:46:44.047 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-08-11 15:46:30.481942336 +0000 2023-08-11T15:46:44.047 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-08-11 15:44:53.723625099 +0000 2023-08-11T15:46:44.047 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-08-11T15:46:44.047 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-08-11T15:46:44.134 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T15:46:44.186 INFO:tasks.workunit.client.0.smithi032.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-08-11T15:46:44.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:44.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:44.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-08-11T15:46:44.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: from='client.25388 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:44.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:44 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-08-11T15:46:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: from='client.25388 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:44.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:44 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:44.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:44.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-08-11T15:46:44.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: from='client.25388 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:44.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:44.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:44 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:45 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:45 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:46:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:45.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:45.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:45 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:46.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-08-11T15:46:46.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-11T15:46:46.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-08-11T15:46:46.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-08-11T15:46:46.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:46.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:46.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:46.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:46 smithi192 ceph-mon[130806]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-08-11T15:46:46.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-08-11T15:46:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-11T15:46:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-08-11T15:46:46.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-08-11T15:46:46.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:46.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:46.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:46.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:46 smithi134 ceph-mon[130447]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-08-11T15:46:46.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-08-11T15:46:46.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-11T15:46:46.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-08-11T15:46:46.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-08-11T15:46:46.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:46.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:46.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:46.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:46 smithi032 ceph-mon[97103]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-08-11T15:46:47.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:47.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:47.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:47.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:47.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:47.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:47.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi032 to 222.0M 2023-08-11T15:46:47.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Unable to set osd_memory_target on smithi032 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T15:46:47.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:46:47.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:46:47.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi032:/etc/ceph/ceph.conf 2023-08-11T15:46:47.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi134:/etc/ceph/ceph.conf 2023-08-11T15:46:47.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:46:47.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:47.973 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:47.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:47.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:47.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:47.974 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:47 smithi134 ceph-mon[130447]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:48.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:48.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:48.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi032 to 222.0M 2023-08-11T15:46:48.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Unable to set osd_memory_target on smithi032 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T15:46:48.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:46:48.055 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:46:48.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi032:/etc/ceph/ceph.conf 2023-08-11T15:46:48.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi134:/etc/ceph/ceph.conf 2023-08-11T15:46:48.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:46:48.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:48.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:48.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:48.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:48.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:48.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:47 smithi032 ceph-mon[97103]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:48.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:48.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:48.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:46:48.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi032 to 222.0M 2023-08-11T15:46:48.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Unable to set osd_memory_target on smithi032 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T15:46:48.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:46:48.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:46:48.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi032:/etc/ceph/ceph.conf 2023-08-11T15:46:48.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi134:/etc/ceph/ceph.conf 2023-08-11T15:46:48.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi192:/etc/ceph/ceph.conf 2023-08-11T15:46:48.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:48.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:48.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:46:48.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:48.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:48.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:47 smithi192 ceph-mon[130806]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T15:46:49.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:49.054 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:48 smithi032 ceph-mon[97103]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:49.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:48 smithi192 ceph-mon[130806]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: Updating smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: Updating smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:49.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:49.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2023-08-11T15:46:49.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:48 smithi134 ceph-mon[130447]: Updating smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T15:46:50.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:50.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:46:50.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:46:50.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:49 smithi192 ceph-mon[130806]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 132 KiB/s wr, 65 op/s 2023-08-11T15:46:50.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:50.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:46:50.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:46:50.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.222 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:49 smithi134 ceph-mon[130447]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 132 KiB/s wr, 65 op/s 2023-08-11T15:46:50.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:46:50.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:46:50.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:46:50.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:46:50.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:49 smithi032 ceph-mon[97103]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 132 KiB/s wr, 65 op/s 2023-08-11T15:46:52.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:52 smithi134 ceph-mon[130447]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 103 KiB/s wr, 50 op/s 2023-08-11T15:46:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:52 smithi032 ceph-mon[97103]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 103 KiB/s wr, 50 op/s 2023-08-11T15:46:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:52 smithi192 ceph-mon[130806]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 103 KiB/s wr, 50 op/s 2023-08-11T15:46:54.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:54 smithi134 ceph-mon[130447]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 84 KiB/s wr, 42 op/s 2023-08-11T15:46:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:54 smithi032 ceph-mon[97103]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 84 KiB/s wr, 42 op/s 2023-08-11T15:46:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:54 smithi192 ceph-mon[130806]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 84 KiB/s wr, 42 op/s 2023-08-11T15:46:56.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:56 smithi032 ceph-mon[97103]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 38 op/s 2023-08-11T15:46:56.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:56 smithi192 ceph-mon[130806]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 38 op/s 2023-08-11T15:46:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:56 smithi134 ceph-mon[130447]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 38 op/s 2023-08-11T15:46:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.241+0000 7f2cadfff700 1 -- 172.21.15.32:0/2803765298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca8073670 msgr2=0x7f2ca810cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.241+0000 7f2cadfff700 1 --2- 172.21.15.32:0/2803765298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca8073670 0x7f2ca810cbb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0098f0 tx=0x7f2c9c009c00 comp rx=0 tx=0).stop 2023-08-11T15:46:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.242+0000 7f2cadfff700 1 -- 172.21.15.32:0/2803765298 shutdown_connections 2023-08-11T15:46:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.242+0000 7f2cadfff700 1 --2- 172.21.15.32:0/2803765298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca8072d50 0x7f2ca8073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.242+0000 7f2cadfff700 1 --2- 172.21.15.32:0/2803765298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ca810d0f0 0x7f2ca810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.242+0000 7f2cadfff700 1 --2- 172.21.15.32:0/2803765298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca8073670 0x7f2ca810cbb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.242+0000 7f2cadfff700 1 -- 172.21.15.32:0/2803765298 >> 172.21.15.32:0/2803765298 conn(0x7f2ca80fc380 msgr2=0x7f2ca80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.242+0000 7f2cadfff700 1 -- 172.21.15.32:0/2803765298 shutdown_connections 2023-08-11T15:46:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.242+0000 7f2cadfff700 1 -- 172.21.15.32:0/2803765298 wait complete. 2023-08-11T15:46:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 Processor -- start 2023-08-11T15:46:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 -- start start 2023-08-11T15:46:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ca8072d50 0x7f2ca819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca8073670 0x7f2ca819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca810d0f0 0x7f2ca8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8199100 con 0x7f2ca8073670 2023-08-11T15:46:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8199270 con 0x7f2ca810d0f0 2023-08-11T15:46:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2cadfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8199430 con 0x7f2ca8072d50 2023-08-11T15:46:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca8073670 0x7f2ca819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.243+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ca8072d50 0x7f2ca819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ca8072d50 0x7f2ca819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:52286/0 (socket says 172.21.15.32:52286) 2023-08-11T15:46:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca77fe700 1 -- 172.21.15.32:0/3958526854 learned_addr learned my addr 172.21.15.32:0/3958526854 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca7fff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca810d0f0 0x7f2ca8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/3958526854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ca8072d50 msgr2=0x7f2ca819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ca8072d50 0x7f2ca819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/3958526854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca810d0f0 msgr2=0x7f2ca8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca810d0f0 0x7f2ca8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/3958526854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980095f0 con 0x7f2ca8073670 2023-08-11T15:46:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca8073670 0x7f2ca819f170 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009810 tx=0x7f2c9c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.244+0000 7f2ca4ff9700 1 -- 172.21.15.32:0/3958526854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c01c070 con 0x7f2ca8073670 2023-08-11T15:46:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.245+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c009520 con 0x7f2ca8073670 2023-08-11T15:46:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.245+0000 7f2ca4ff9700 1 -- 172.21.15.32:0/3958526854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c00bba0 con 0x7f2ca8073670 2023-08-11T15:46:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.245+0000 7f2ca4ff9700 1 -- 172.21.15.32:0/3958526854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c003970 con 0x7f2ca8073670 2023-08-11T15:46:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.245+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca81a5a40 con 0x7f2ca8073670 2023-08-11T15:46:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.248+0000 7f2ca4ff9700 1 -- 172.21.15.32:0/3958526854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) v1 ==== 134847+0+0 (secure 0 0 0) 0x7f2c9c020030 con 0x7f2ca8073670 2023-08-11T15:46:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.249+0000 7f2cc19b0b80 1 Processor -- start 2023-08-11T15:46:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.249+0000 7f2cc19b0b80 1 -- start start 2023-08-11T15:46:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bfde80 0x55ee42bfe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8630 0x55ee42bf8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c04c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42b515d0 con 0x55ee42bf8ff0 2023-08-11T15:46:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42b4eef0 con 0x55ee42bf8630 2023-08-11T15:46:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42b4a7e0 con 0x55ee42bfde80 2023-08-11T15:46:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca4ff9700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2c900a0150 0x7f2c900a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c04c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c04c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54704/0 (socket says 172.21.15.32:54704) 2023-08-11T15:46:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8630 0x55ee42bf8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/2953293960 learned_addr learned my addr 172.21.15.32:0/2953293960 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca7fff700 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bfde80 0x55ee42bfe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca4ff9700 1 -- 172.21.15.32:0/3958526854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2c9c0cc910 con 0x7f2ca8073670 2023-08-11T15:46:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.250+0000 7f2ca7fff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2c900a0150 0x7f2c900a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/2953293960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bfde80 msgr2=0x55ee42bfe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bfde80 0x55ee42bfe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/2953293960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8630 msgr2=0x55ee42bf8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8630 0x55ee42bf8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/2953293960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee429ca5d0 con 0x55ee42bf8ff0 2023-08-11T15:46:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8630 0x55ee42bf8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c04c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0c9ba0 tx=0x7f2c9c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551ba9acc1d3f0e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2c8dffb700 1 -- 172.21.15.32:0/2953293960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c09a2d0 con 0x55ee42bf8ff0 2023-08-11T15:46:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2ca7fff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2c900a0150 0x7f2c900a2610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009ec0 tx=0x7f2ca000b040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2c8dffb700 1 -- 172.21.15.32:0/2953293960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c021380 con 0x55ee42bf8ff0 2023-08-11T15:46:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.251+0000 7f2c8dffb700 1 -- 172.21.15.32:0/2953293960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c005260 con 0x55ee42bf8ff0 2023-08-11T15:46:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2953293960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 msgr2=0x55ee42c04c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c04c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0c9ba0 tx=0x7f2c9c009f50 comp rx=0 tx=0).stop 2023-08-11T15:46:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2953293960 shutdown_connections 2023-08-11T15:46:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c04c10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bfde80 0x55ee42bfe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2953293960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8630 0x55ee42bf8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2953293960 >> 172.21.15.32:0/2953293960 conn(0x55ee42b258f0 msgr2=0x55ee42bf2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2953293960 shutdown_connections 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.252+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2953293960 wait complete. 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.253+0000 7f2cc19b0b80 1 -- start start 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.253+0000 7f2cc19b0b80 1 Processor -- start 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bfde80 0x55ee42c018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42ae8a60 con 0x55ee42bfde80 2023-08-11T15:46:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42ae8760 con 0x55ee42bf8ff0 2023-08-11T15:46:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee429cc8c0 con 0x55ee42bf8630 2023-08-11T15:46:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2cc19b0b80 10 client.0 _ll_register_callbacks cb 0x55ee42aeba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:46:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2ca7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bfde80 0x55ee42c018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:52302/0 (socket says 172.21.15.32:52302) 2023-08-11T15:46:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2ca77fe700 1 -- 172.21.15.32:0/1918150263 learned_addr learned my addr 172.21.15.32:0/1918150263 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.254+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2ca77fe700 1 -- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8ff0 msgr2=0x55ee42c07ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2ca77fe700 1 -- 172.21.15.32:0/1918150263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bfde80 msgr2=0x55ee42c018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bfde80 0x55ee42c018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2ca77fe700 1 -- 172.21.15.32:0/1918150263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c98009c10 con 0x55ee42bf8630 2023-08-11T15:46:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2ca7fff700 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bfde80 0x55ee42c018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c075a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c98012d20 tx=0x7f2c9800f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0374849b9fc4a48 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2c86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:46:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2c86ffd700 1 -- 172.21.15.32:0/1918150263 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98017070 con 0x55ee42bf8630 2023-08-11T15:46:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee42beb7e0 con 0x55ee42bf8630 2023-08-11T15:46:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.255+0000 7f2c86ffd700 1 -- 172.21.15.32:0/1918150263 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2c9800fb80 con 0x55ee42bf8630 2023-08-11T15:46:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2c86ffd700 1 -- 172.21.15.32:0/1918150263 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c9800fd40 con 0x55ee42bf8630 2023-08-11T15:46:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 10 client.25411 fetch_fsmap learned FSMap version 21 2023-08-11T15:46:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7f2ca81a5a40 con 0x55ee42bf8630 2023-08-11T15:46:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2c86ffd700 1 -- 172.21.15.32:0/1918150263 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee42beb7e0 con 0x55ee42bf8630 2023-08-11T15:46:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2c86ffd700 1 -- 172.21.15.32:0/1918150263 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c98003b80 con 0x55ee42bf8630 2023-08-11T15:46:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 10 client.25411 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:46:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 10 client.25411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:46:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 20 client.25411 populate_metadata read hostname 'smithi032' 2023-08-11T15:46:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c11af0 0x55ee42c13f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 4 client.25411 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:46:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.256+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee42bc7480 con 0x55ee42c11af0 2023-08-11T15:46:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.257+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c11af0 0x55ee42c13f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.257+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c11af0 0x55ee42c13f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77bb62c2ffb6186d server_cookie=c205aaf4fc222154 in_seq=0 out_seq=0 2023-08-11T15:46:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.257+0000 7f2c86ffd700 10 client.25411 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2c86ffd700 1 -- 172.21.15.32:0/1918150263 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55ee42bc7480 con 0x55ee42c11af0 2023-08-11T15:46:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2c86ffd700 10 client.25411 handle_command_reply: tid=0 2023-08-11T15:46:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 client.25411 shutdown 2023-08-11T15:46:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 msgr2=0x55ee42c075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c075a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c98012d20 tx=0x7f2c9800f6a0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 shutdown_connections 2023-08-11T15:46:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c11af0 0x55ee42c13f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bfde80 0x55ee42c018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c075a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/1918150263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2c86ffd700 0 client.25411 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 >> 172.21.15.32:0/1918150263 conn(0x55ee42b258f0 msgr2=0x55ee42bf7620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 shutdown_connections 2023-08-11T15:46:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.258+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/1918150263 wait complete. 2023-08-11T15:46:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.259+0000 7f2cc19b0b80 20 client.25411 trim_cache size 0 max 16384 2023-08-11T15:46:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.274+0000 7f2cc19b0b80 1 Processor -- start 2023-08-11T15:46:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.274+0000 7f2cc19b0b80 1 -- start start 2023-08-11T15:46:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42bfd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42bf2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42c0f1a0 con 0x55ee42bf8ff0 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42bf41a0 con 0x55ee42bfde80 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42beb570 con 0x55ee42bf8630 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42bfd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54718/0 (socket says 172.21.15.32:54718) 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca6ffd700 1 -- 172.21.15.32:0/2646265634 learned_addr learned my addr 172.21.15.32:0/2646265634 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca7fff700 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42bf2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca77fe700 1 -- 172.21.15.32:0/2646265634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 msgr2=0x55ee42bf2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42bf2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca77fe700 1 -- 172.21.15.32:0/2646265634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 msgr2=0x55ee42c07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.275+0000 7f2ca77fe700 1 -- 172.21.15.32:0/2646265634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee42befa10 con 0x55ee42bf8630 2023-08-11T15:46:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.276+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42bfd200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c98003fb0 tx=0x7f2c9800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4936b8b42a56d1ba server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.276+0000 7f2c8e7fc700 1 -- 172.21.15.32:0/2646265634 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98017040 con 0x55ee42bf8630 2023-08-11T15:46:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.276+0000 7f2c8e7fc700 1 -- 172.21.15.32:0/2646265634 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c98010720 con 0x55ee42bf8630 2023-08-11T15:46:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.276+0000 7f2c8e7fc700 1 -- 172.21.15.32:0/2646265634 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98003aa0 con 0x55ee42bf8630 2023-08-11T15:46:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2646265634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 msgr2=0x55ee42bfd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42bfd200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c98003fb0 tx=0x7f2c9800fa70 comp rx=0 tx=0).stop 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2646265634 shutdown_connections 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42bf2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42bfd200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/2646265634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2646265634 >> 172.21.15.32:0/2646265634 conn(0x55ee42b258f0 msgr2=0x55ee42bf7620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2646265634 shutdown_connections 2023-08-11T15:46:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/2646265634 wait complete. 2023-08-11T15:46:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 -- start start 2023-08-11T15:46:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.277+0000 7f2cc19b0b80 1 Processor -- start 2023-08-11T15:46:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2cc19b0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42c64780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42c64cc0 con 0x55ee42bf8ff0 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42c64e00 con 0x55ee42bfde80 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2cc19b0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee42c64fc0 con 0x55ee42bf8630 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2cc19b0b80 10 client.0 _ll_register_callbacks cb 0x7f2c980127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:52316/0 (socket says 172.21.15.32:52316) 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2ca77fe700 1 -- 172.21.15.32:0/3035982176 learned_addr learned my addr 172.21.15.32:0/3035982176 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:46:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.278+0000 7f2ca7fff700 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42c64780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2ca77fe700 1 -- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 msgr2=0x55ee42c64780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42c64780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2ca77fe700 1 -- 172.21.15.32:0/3035982176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 msgr2=0x55ee42c055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:46:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2ca77fe700 1 -- 172.21.15.32:0/3035982176 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c98021040 con 0x55ee42bf8630 2023-08-11T15:46:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:46:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2ca77fe700 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c050b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c98010450 tx=0x7f2c98009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98572e4725e45efa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:46:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2c86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:46:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2c86ffd700 1 -- 172.21.15.32:0/3035982176 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98017070 con 0x55ee42bf8630 2023-08-11T15:46:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c9c021380 con 0x55ee42bf8630 2023-08-11T15:46:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2c86ffd700 1 -- 172.21.15.32:0/3035982176 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2c98010520 con 0x55ee42bf8630 2023-08-11T15:46:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2c86ffd700 1 -- 172.21.15.32:0/3035982176 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c98010890 con 0x55ee42bf8630 2023-08-11T15:46:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2cc19b0b80 10 client.25417 fetch_fsmap learned FSMap version 21 2023-08-11T15:46:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.279+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ee42976d80 con 0x55ee42bf8630 2023-08-11T15:46:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2c86ffd700 1 -- 172.21.15.32:0/3035982176 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c021380 con 0x55ee42bf8630 2023-08-11T15:46:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2c86ffd700 1 -- 172.21.15.32:0/3035982176 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c98030070 con 0x55ee42bf8630 2023-08-11T15:46:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2cc19b0b80 10 client.25417 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:46:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2cc19b0b80 10 client.25417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:46:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2cc19b0b80 20 client.25417 populate_metadata read hostname 'smithi032' 2023-08-11T15:46:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c66390 0x55ee42c68780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:46:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2cc19b0b80 4 client.25417 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:46:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee42b437c0 con 0x55ee42c66390 2023-08-11T15:46:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.280+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c66390 0x55ee42c68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:46:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.281+0000 7f2c86ffd700 10 client.25417 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.281+0000 7f2ca6ffd700 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c66390 0x55ee42c68780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8fafc79c7c2e8bc server_cookie=50fe903f2d6748e3 in_seq=0 out_seq=0 2023-08-11T15:46:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.294+0000 7f2c86ffd700 1 -- 172.21.15.32:0/3035982176 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee42b437c0 con 0x55ee42c66390 2023-08-11T15:46:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.294+0000 7f2c86ffd700 10 client.25417 handle_command_reply: tid=0 2023-08-11T15:46:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 client.25417 shutdown 2023-08-11T15:46:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 msgr2=0x55ee42c050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c050b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c98010450 tx=0x7f2c98009a10 comp rx=0 tx=0).stop 2023-08-11T15:46:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 shutdown_connections 2023-08-11T15:46:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55ee42c66390 0x55ee42c68780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee42bfde80 0x55ee42c64780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee42bf8630 0x55ee42c050b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2c86ffd700 0 client.25417 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:46:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 --2- 172.21.15.32:0/3035982176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee42bf8ff0 0x55ee42c055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 >> 172.21.15.32:0/3035982176 conn(0x55ee42b258f0 msgr2=0x55ee42bf7620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 shutdown_connections 2023-08-11T15:46:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 1 -- 172.21.15.32:0/3035982176 wait complete. 2023-08-11T15:46:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.295+0000 7f2cc19b0b80 20 client.25417 trim_cache size 0 max 16384 2023-08-11T15:46:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:46:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:46:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:46:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:46:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2c900a0150 msgr2=0x7f2c900a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2c900a0150 0x7f2c900a2610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009ec0 tx=0x7f2ca000b040 comp rx=0 tx=0).stop 2023-08-11T15:46:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca8073670 msgr2=0x7f2ca819f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:46:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca8073670 0x7f2ca819f170 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009810 tx=0x7f2c9c00fa00 comp rx=0 tx=0).stop 2023-08-11T15:46:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 shutdown_connections 2023-08-11T15:46:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2c900a0150 0x7f2c900a2610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ca810d0f0 0x7f2ca8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ca8072d50 0x7f2ca819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 --2- 172.21.15.32:0/3958526854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca8073670 0x7f2ca819f170 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:46:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 >> 172.21.15.32:0/3958526854 conn(0x7f2ca80fc380 msgr2=0x7f2ca810f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:46:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 shutdown_connections 2023-08-11T15:46:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:46:57.296+0000 7f2cadfff700 1 -- 172.21.15.32:0/3958526854 wait complete. 2023-08-11T15:46:58.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:46:57 smithi134 ceph-mon[130447]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 38 op/s 2023-08-11T15:46:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:46:57 smithi032 ceph-mon[97103]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 38 op/s 2023-08-11T15:46:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:46:57 smithi192 ceph-mon[130806]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 38 op/s 2023-08-11T15:47:00.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:00 smithi134 ceph-mon[130447]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 39 op/s 2023-08-11T15:47:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:00 smithi032 ceph-mon[97103]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 39 op/s 2023-08-11T15:47:00.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:00 smithi192 ceph-mon[130806]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 77 KiB/s wr, 39 op/s 2023-08-11T15:47:01.303 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:47:00 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:47:00.814+0000 7f335ad61700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:47:01.304 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:47:00 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:47:00.814+0000 7f335ad61700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:47:01.304 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:47:00 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:47:00.814+0000 7f335ad61700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:47:01.304 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:47:00 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:47:00.814+0000 7f335ad61700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:47:01.304 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:47:00 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:47:00.814+0000 7f335ad61700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T15:47:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:02 smithi032 ceph-mon[97103]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:47:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:02 smithi032 ceph-mon[97103]: mgrmap e25: x(active, since 20s), standbys: z, y 2023-08-11T15:47:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:02 smithi192 ceph-mon[130806]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:47:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:02 smithi192 ceph-mon[130806]: mgrmap e25: x(active, since 20s), standbys: z, y 2023-08-11T15:47:02.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:02 smithi134 ceph-mon[130447]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:47:02.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:02 smithi134 ceph-mon[130447]: mgrmap e25: x(active, since 20s), standbys: z, y 2023-08-11T15:47:04.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:04 smithi032 ceph-mon[97103]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-08-11T15:47:04.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:04 smithi192 ceph-mon[130806]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-08-11T15:47:04.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:04 smithi134 ceph-mon[130447]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-08-11T15:47:06.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:06 smithi032 ceph-mon[97103]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-08-11T15:47:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:06 smithi192 ceph-mon[130806]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-08-11T15:47:06.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:06 smithi134 ceph-mon[130447]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-08-11T15:47:08.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:07 smithi134 ceph-mon[130447]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-08-11T15:47:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:07 smithi032 ceph-mon[97103]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-08-11T15:47:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:07 smithi192 ceph-mon[130806]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-08-11T15:47:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 -- 172.21.15.32:0/1434928116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1068b0 msgr2=0x7f646c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 --2- 172.21.15.32:0/1434928116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1068b0 0x7f646c106c90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f645c009990 tx=0x7f645c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 -- 172.21.15.32:0/1434928116 shutdown_connections 2023-08-11T15:47:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 --2- 172.21.15.32:0/1434928116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1068b0 0x7f646c106c90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 --2- 172.21.15.32:0/1434928116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c101260 0x7f646c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 --2- 172.21.15.32:0/1434928116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f646c1008a0 0x7f646c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 -- 172.21.15.32:0/1434928116 >> 172.21.15.32:0/1434928116 conn(0x7f646c0fc370 msgr2=0x7f646c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 -- 172.21.15.32:0/1434928116 shutdown_connections 2023-08-11T15:47:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.451+0000 7f64711ff700 1 -- 172.21.15.32:0/1434928116 wait complete. 2023-08-11T15:47:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.453+0000 7f64711ff700 1 Processor -- start 2023-08-11T15:47:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f64711ff700 1 -- start start 2023-08-11T15:47:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f64711ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1008a0 0x7f646c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f64711ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c101260 0x7f646c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f64711ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f646c1068b0 0x7f646c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f64711ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c1990c0 con 0x7f646c1008a0 2023-08-11T15:47:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f64711ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c199230 con 0x7f646c1068b0 2023-08-11T15:47:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f64711ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c1993f0 con 0x7f646c101260 2023-08-11T15:47:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.454+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1008a0 0x7f646c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1008a0 0x7f646c19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50416/0 (socket says 172.21.15.32:50416) 2023-08-11T15:47:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 -- 172.21.15.32:0/106870110 learned_addr learned my addr 172.21.15.32:0/106870110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646b59e700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f646c1068b0 0x7f646c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646a59c700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c101260 0x7f646c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 -- 172.21.15.32:0/106870110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c101260 msgr2=0x7f646c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c101260 0x7f646c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 -- 172.21.15.32:0/106870110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f646c1068b0 msgr2=0x7f646c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f646c1068b0 0x7f646c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 -- 172.21.15.32:0/106870110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0095f0 con 0x7f646c1008a0 2023-08-11T15:47:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.455+0000 7f646ad9d700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1008a0 0x7f646c19eb20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f645c005910 tx=0x7f645c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.456+0000 7f6453fff700 1 -- 172.21.15.32:0/106870110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c01d070 con 0x7f646c1008a0 2023-08-11T15:47:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.456+0000 7f6453fff700 1 -- 172.21.15.32:0/106870110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f645c022470 con 0x7f646c1008a0 2023-08-11T15:47:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.456+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c199670 con 0x7f646c1008a0 2023-08-11T15:47:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.456+0000 7f6453fff700 1 -- 172.21.15.32:0/106870110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c017650 con 0x7f646c1008a0 2023-08-11T15:47:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.456+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c1a5aa0 con 0x7f646c1008a0 2023-08-11T15:47:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.459+0000 7f6453fff700 1 -- 172.21.15.32:0/106870110 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7f645c00fad0 con 0x7f646c1008a0 2023-08-11T15:47:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 Processor -- start 2023-08-11T15:47:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 -- start start 2023-08-11T15:47:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17ea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab173d5d0 con 0x55fab17e9e80 2023-08-11T15:47:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab173aef0 con 0x55fab17e4630 2023-08-11T15:47:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.461+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab17367e0 con 0x55fab17e4ff0 2023-08-11T15:47:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f6453fff700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64540a01e0 0x7f64540a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17e4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:38020/0 (socket says 172.21.15.32:38020) 2023-08-11T15:47:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646ad9d700 1 -- 172.21.15.32:0/1357421674 learned_addr learned my addr 172.21.15.32:0/1357421674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646a59c700 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646b59e700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64540a01e0 0x7f64540a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f6453fff700 1 -- 172.21.15.32:0/106870110 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f645c0cd0e0 con 0x7f646c1008a0 2023-08-11T15:47:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646a59c700 1 -- 172.21.15.32:0/1357421674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 msgr2=0x55fab17e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646a59c700 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646a59c700 1 -- 172.21.15.32:0/1357421674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 msgr2=0x55fab17ea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646a59c700 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17ea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646a59c700 1 -- 172.21.15.32:0/1357421674 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab15b65d0 con 0x55fab17e4ff0 2023-08-11T15:47:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.462+0000 7f646ad9d700 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:47:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.463+0000 7f646a59c700 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f0c10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f646c19a3c0 tx=0x7f646000af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e6fa7948f30643 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.463+0000 7f646b59e700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64540a01e0 0x7f64540a26a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6464009ec0 tx=0x7f646400b040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.463+0000 7f64517fa700 1 -- 172.21.15.32:0/1357421674 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646000fd60 con 0x55fab17e4ff0 2023-08-11T15:47:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.463+0000 7f64517fa700 1 -- 172.21.15.32:0/1357421674 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f646000fec0 con 0x55fab17e4ff0 2023-08-11T15:47:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.463+0000 7f64517fa700 1 -- 172.21.15.32:0/1357421674 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460005280 con 0x55fab17e4ff0 2023-08-11T15:47:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1357421674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 msgr2=0x55fab17f0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f0c10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f646c19a3c0 tx=0x7f646000af10 comp rx=0 tx=0).stop 2023-08-11T15:47:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1357421674 shutdown_connections 2023-08-11T15:47:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f0c10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1357421674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1357421674 >> 172.21.15.32:0/1357421674 conn(0x55fab17118f0 msgr2=0x55fab17dee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1357421674 shutdown_connections 2023-08-11T15:47:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.464+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1357421674 wait complete. 2023-08-11T15:47:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.465+0000 7f6484bb9b80 1 -- start start 2023-08-11T15:47:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.465+0000 7f6484bb9b80 1 Processor -- start 2023-08-11T15:47:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.465+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4630 0x55fab17f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.465+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab16d4a60 con 0x55fab17e4630 2023-08-11T15:47:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab16d4760 con 0x55fab17e9e80 2023-08-11T15:47:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab15b88c0 con 0x55fab17e4ff0 2023-08-11T15:47:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f6484bb9b80 10 client.0 _ll_register_callbacks cb 0x55fab16d7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4630 0x55fab17f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4630 0x55fab17f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50442/0 (socket says 172.21.15.32:50442) 2023-08-11T15:47:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 -- 172.21.15.32:0/1721510621 learned_addr learned my addr 172.21.15.32:0/1721510621 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646b59e700 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646a59c700 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 -- 172.21.15.32:0/1721510621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 msgr2=0x55fab17f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 -- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 msgr2=0x55fab17edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.466+0000 7f646ad9d700 1 -- 172.21.15.32:0/1721510621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c1a5aa0 con 0x55fab17e4630 2023-08-11T15:47:10.474 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:10 smithi134 ceph-mon[130447]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 25 KiB/s wr, 19 op/s 2023-08-11T15:47:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f646ad9d700 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4630 0x55fab17f39f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f645c005f50 tx=0x7f645c06c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9997a7b105ac8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f644eff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:47:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f644eff5700 1 -- 172.21.15.32:0/1721510621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c099c20 con 0x55fab17e4630 2023-08-11T15:47:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fab17b3480 con 0x55fab17e4630 2023-08-11T15:47:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f644eff5700 1 -- 172.21.15.32:0/1721510621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f645c066380 con 0x55fab17e4630 2023-08-11T15:47:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f644eff5700 1 -- 172.21.15.32:0/1721510621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f645c09a9c0 con 0x55fab17e4630 2023-08-11T15:47:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f6484bb9b80 10 client.15423 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7f646000a890 con 0x55fab17e4630 2023-08-11T15:47:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.467+0000 7f644eff5700 1 -- 172.21.15.32:0/1721510621 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fab17b3480 con 0x55fab17e4630 2023-08-11T15:47:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f644eff5700 1 -- 172.21.15.32:0/1721510621 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f645c06c160 con 0x55fab17e4630 2023-08-11T15:47:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f6484bb9b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f6484bb9b80 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f6484bb9b80 20 client.15423 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab17fdb70 0x55fab17fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f6484bb9b80 4 client.15423 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fab172f7c0 con 0x55fab17fdb70 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.468+0000 7f646a59c700 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab17fdb70 0x55fab17fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.469+0000 7f646a59c700 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab17fdb70 0x55fab17fff60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe629b3efdf5f448 server_cookie=76ac27f3e6372a81 in_seq=0 out_seq=0 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.469+0000 7f644eff5700 10 client.15423 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f644eff5700 1 -- 172.21.15.32:0/1721510621 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55fab172f7c0 con 0x55fab17fdb70 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f644eff5700 10 client.15423 handle_command_reply: tid=0 2023-08-11T15:47:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 client.15423 shutdown 2023-08-11T15:47:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4630 msgr2=0x55fab17f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4630 0x55fab17f39f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f645c005f50 tx=0x7f645c06c450 comp rx=0 tx=0).stop 2023-08-11T15:47:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 shutdown_connections 2023-08-11T15:47:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab17fdb70 0x55fab17fff60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4630 0x55fab17f39f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/1721510621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f644eff5700 0 client.15423 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 >> 172.21.15.32:0/1721510621 conn(0x55fab17118f0 msgr2=0x55fab17de5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 shutdown_connections 2023-08-11T15:47:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 1 -- 172.21.15.32:0/1721510621 wait complete. 2023-08-11T15:47:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.470+0000 7f6484bb9b80 20 client.15423 trim_cache size 0 max 16384 2023-08-11T15:47:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.487+0000 7f6484bb9b80 1 Processor -- start 2023-08-11T15:47:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.487+0000 7f6484bb9b80 1 -- start start 2023-08-11T15:47:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4630 0x55fab17ded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4ff0 0x55fab17e6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab17df7d0 con 0x55fab17e4ff0 2023-08-11T15:47:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab17e0de0 con 0x55fab17e9e80 2023-08-11T15:47:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab17e5490 con 0x55fab17e4630 2023-08-11T15:47:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4ff0 0x55fab17e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646b59e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4ff0 0x55fab17e6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50450/0 (socket says 172.21.15.32:50450) 2023-08-11T15:47:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646a59c700 1 -- 172.21.15.32:0/4051922640 learned_addr learned my addr 172.21.15.32:0/4051922640 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646ad9d700 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4630 0x55fab17ded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646b59e700 1 -- 172.21.15.32:0/4051922640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4630 msgr2=0x55fab17ded50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646b59e700 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4630 0x55fab17ded50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646b59e700 1 -- 172.21.15.32:0/4051922640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4ff0 msgr2=0x55fab17e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646b59e700 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4ff0 0x55fab17e6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.488+0000 7f646b59e700 1 -- 172.21.15.32:0/4051922640 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab1562d80 con 0x55fab17e9e80 2023-08-11T15:47:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.489+0000 7f646a59c700 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4ff0 0x55fab17e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:47:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.489+0000 7f646b59e700 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17e6da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64640089f0 tx=0x7f6464010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b3c34962e88bda server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.489+0000 7f6451ffb700 1 -- 172.21.15.32:0/4051922640 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464022050 con 0x55fab17e9e80 2023-08-11T15:47:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.489+0000 7f6451ffb700 1 -- 172.21.15.32:0/4051922640 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6464023040 con 0x55fab17e9e80 2023-08-11T15:47:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.489+0000 7f6451ffb700 1 -- 172.21.15.32:0/4051922640 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646400e070 con 0x55fab17e9e80 2023-08-11T15:47:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 -- 172.21.15.32:0/4051922640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 msgr2=0x55fab17e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17e6da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64640089f0 tx=0x7f6464010000 comp rx=0 tx=0).stop 2023-08-11T15:47:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 -- 172.21.15.32:0/4051922640 shutdown_connections 2023-08-11T15:47:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e9e80 0x55fab17e6da0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4630 0x55fab17ded50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/4051922640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e4ff0 0x55fab17e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 -- 172.21.15.32:0/4051922640 >> 172.21.15.32:0/4051922640 conn(0x55fab17118f0 msgr2=0x55fab17e7390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 -- 172.21.15.32:0/4051922640 shutdown_connections 2023-08-11T15:47:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 -- 172.21.15.32:0/4051922640 wait complete. 2023-08-11T15:47:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.490+0000 7f6484bb9b80 1 -- start start 2023-08-11T15:47:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 1 Processor -- start 2023-08-11T15:47:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab17f20f0 con 0x55fab17e9e80 2023-08-11T15:47:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab17f2260 con 0x55fab17e4630 2023-08-11T15:47:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab17f2420 con 0x55fab17e4ff0 2023-08-11T15:47:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.491+0000 7f6484bb9b80 10 client.0 _ll_register_callbacks cb 0x55fab16d7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:38064/0 (socket says 172.21.15.32:38064) 2023-08-11T15:47:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 -- 172.21.15.32:0/3269161096 learned_addr learned my addr 172.21.15.32:0/3269161096 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50458/0 (socket says 172.21.15.32:50458) 2023-08-11T15:47:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646a59c700 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 -- 172.21.15.32:0/3269161096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 msgr2=0x55fab17f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 -- 172.21.15.32:0/3269161096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 msgr2=0x55fab17f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 -- 172.21.15.32:0/3269161096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fab15b65d0 con 0x55fab17e4630 2023-08-11T15:47:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f646ad9d700 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17f3320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f645c022bd0 tx=0x7f645c096730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2955286089b817 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.492+0000 7f644eff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:47:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f644eff5700 1 -- 172.21.15.32:0/3269161096 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c09a720 con 0x55fab17e4630 2023-08-11T15:47:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fab17d77e0 con 0x55fab17e4630 2023-08-11T15:47:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f644eff5700 1 -- 172.21.15.32:0/3269161096 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f645c09a9c0 con 0x55fab17e4630 2023-08-11T15:47:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f644eff5700 1 -- 172.21.15.32:0/3269161096 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f645c09a090 con 0x55fab17e4630 2023-08-11T15:47:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 10 client.25436 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7f6464003a00 con 0x55fab17e4630 2023-08-11T15:47:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f644eff5700 1 -- 172.21.15.32:0/3269161096 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fab17d77e0 con 0x55fab17e4630 2023-08-11T15:47:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f644eff5700 1 -- 172.21.15.32:0/3269161096 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f645c0b2240 con 0x55fab17e4630 2023-08-11T15:47:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 10 client.25436 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 10 client.25436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 20 client.25436 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab1852a10 0x55fab1854e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 4 client.25436 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:47:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.493+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fab16efff0 con 0x55fab1852a10 2023-08-11T15:47:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.494+0000 7f646a59c700 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab1852a10 0x55fab1854e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.494+0000 7f646a59c700 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab1852a10 0x55fab1854e30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02343da95edc23c server_cookie=5da8664abf075959 in_seq=0 out_seq=0 2023-08-11T15:47:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.494+0000 7f644eff5700 10 client.25436 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f644eff5700 1 -- 172.21.15.32:0/3269161096 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fab16efff0 con 0x55fab1852a10 2023-08-11T15:47:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f644eff5700 10 client.25436 handle_command_reply: tid=0 2023-08-11T15:47:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 client.25436 shutdown 2023-08-11T15:47:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 msgr2=0x55fab17f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17f3320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f645c022bd0 tx=0x7f645c096730 comp rx=0 tx=0).stop 2023-08-11T15:47:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 shutdown_connections 2023-08-11T15:47:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55fab1852a10 0x55fab1854e30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fab17e9e80 0x55fab17f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fab17e4630 0x55fab17f3320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 --2- 172.21.15.32:0/3269161096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fab17e4ff0 0x55fab17f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f644eff5700 0 client.25436 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.502+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 >> 172.21.15.32:0/3269161096 conn(0x55fab17118f0 msgr2=0x55fab17e8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 shutdown_connections 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f6484bb9b80 1 -- 172.21.15.32:0/3269161096 wait complete. 2023-08-11T15:47:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f6484bb9b80 20 client.25436 trim_cache size 0 max 16384 2023-08-11T15:47:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:47:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:47:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:47:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:47:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64540a01e0 msgr2=0x7f64540a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f64711ff700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64540a01e0 0x7f64540a26a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6464009ec0 tx=0x7f646400b040 comp rx=0 tx=0).stop 2023-08-11T15:47:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1008a0 msgr2=0x7f646c19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f64711ff700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1008a0 0x7f646c19eb20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f645c005910 tx=0x7f645c005740 comp rx=0 tx=0).stop 2023-08-11T15:47:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 shutdown_connections 2023-08-11T15:47:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f64711ff700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64540a01e0 0x7f64540a26a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.503+0000 7f64711ff700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646c1008a0 0x7f646c19eb20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.504+0000 7f64711ff700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f646c1068b0 0x7f646c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.504+0000 7f64711ff700 1 --2- 172.21.15.32:0/106870110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c101260 0x7f646c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.504+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 >> 172.21.15.32:0/106870110 conn(0x7f646c0fc370 msgr2=0x7f646c10bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.504+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 shutdown_connections 2023-08-11T15:47:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:10.504+0000 7f64711ff700 1 -- 172.21.15.32:0/106870110 wait complete. 2023-08-11T15:47:10.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:10 smithi032 ceph-mon[97103]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 25 KiB/s wr, 19 op/s 2023-08-11T15:47:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:10 smithi192 ceph-mon[130806]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 25 KiB/s wr, 19 op/s 2023-08-11T15:47:12.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:12 smithi134 ceph-mon[130447]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 25 KiB/s wr, 19 op/s 2023-08-11T15:47:12.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:12 smithi032 ceph-mon[97103]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 25 KiB/s wr, 19 op/s 2023-08-11T15:47:12.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:12 smithi192 ceph-mon[130806]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 25 KiB/s wr, 19 op/s 2023-08-11T15:47:14.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:14 smithi134 ceph-mon[130447]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 KiB/s wr, 22 op/s 2023-08-11T15:47:14.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:14 smithi032 ceph-mon[97103]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 KiB/s wr, 22 op/s 2023-08-11T15:47:14.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:14 smithi192 ceph-mon[130806]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 KiB/s wr, 22 op/s 2023-08-11T15:47:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:16 smithi032 ceph-mon[97103]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:16.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:16 smithi192 ceph-mon[130806]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:16 smithi134 ceph-mon[130447]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:18.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:17 smithi134 ceph-mon[130447]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:17 smithi032 ceph-mon[97103]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:17 smithi192 ceph-mon[130806]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:20.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:20 smithi134 ceph-mon[130447]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2023-08-11T15:47:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:20 smithi032 ceph-mon[97103]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2023-08-11T15:47:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:20 smithi192 ceph-mon[130806]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2023-08-11T15:47:22.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:22 smithi134 ceph-mon[130447]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:22.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:22 smithi032 ceph-mon[97103]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:22.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:22 smithi192 ceph-mon[130806]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 -- 172.21.15.32:0/3003040932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744102bd0 msgr2=0x7f074410d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3003040932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744102bd0 0x7f074410d6e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f073c009f90 tx=0x7f073c00b330 comp rx=0 tx=0).stop 2023-08-11T15:47:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 -- 172.21.15.32:0/3003040932 shutdown_connections 2023-08-11T15:47:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3003040932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744102bd0 0x7f074410d6e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3003040932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0744100ad0 0x7f0744100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3003040932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0744102210 0x7f0744102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 -- 172.21.15.32:0/3003040932 >> 172.21.15.32:0/3003040932 conn(0x7f07440fc1e0 msgr2=0x7f07440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.740+0000 7f0748b9d700 1 -- 172.21.15.32:0/3003040932 shutdown_connections 2023-08-11T15:47:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.741+0000 7f0748b9d700 1 -- 172.21.15.32:0/3003040932 wait complete. 2023-08-11T15:47:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.742+0000 7f0748b9d700 1 Processor -- start 2023-08-11T15:47:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.742+0000 7f0748b9d700 1 -- start start 2023-08-11T15:47:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f0748b9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744100ad0 0x7f074419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f0748b9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0744102210 0x7f074419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f0748b9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0744102bd0 0x7f07441968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f0748b9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744196f10 con 0x7f0744100ad0 2023-08-11T15:47:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f0748b9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744197080 con 0x7f0744102bd0 2023-08-11T15:47:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f0748b9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744197240 con 0x7f0744102210 2023-08-11T15:47:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f07437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744100ad0 0x7f074419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f07437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744100ad0 0x7f074419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54166/0 (socket says 172.21.15.32:54166) 2023-08-11T15:47:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f07437fe700 1 -- 172.21.15.32:0/3553340342 learned_addr learned my addr 172.21.15.32:0/3553340342 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.743+0000 7f0742ffd700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0744102210 0x7f074419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.744+0000 7f0743fff700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0744102bd0 0x7f07441968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.744+0000 7f07437fe700 1 -- 172.21.15.32:0/3553340342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0744102210 msgr2=0x7f074419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.744+0000 7f07437fe700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0744102210 0x7f074419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.744+0000 7f07437fe700 1 -- 172.21.15.32:0/3553340342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0744102bd0 msgr2=0x7f07441968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.744+0000 7f07437fe700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0744102bd0 0x7f07441968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.744+0000 7f07437fe700 1 -- 172.21.15.32:0/3553340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340095f0 con 0x7f0744100ad0 2023-08-11T15:47:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.744+0000 7f07437fe700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744100ad0 0x7f074419c830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f073400ec10 tx=0x7f073400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.745+0000 7f0740ff9700 1 -- 172.21.15.32:0/3553340342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073400cc90 con 0x7f0744100ad0 2023-08-11T15:47:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.745+0000 7f0740ff9700 1 -- 172.21.15.32:0/3553340342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07340044e0 con 0x7f0744100ad0 2023-08-11T15:47:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.745+0000 7f0740ff9700 1 -- 172.21.15.32:0/3553340342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734010430 con 0x7f0744100ad0 2023-08-11T15:47:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.745+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c009c40 con 0x7f0744100ad0 2023-08-11T15:47:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.745+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07441a3640 con 0x7f0744100ad0 2023-08-11T15:47:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.751+0000 7f0740ff9700 1 -- 172.21.15.32:0/3553340342 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7f0734003fb0 con 0x7f0744100ad0 2023-08-11T15:47:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.752+0000 7f075d6feb80 1 Processor -- start 2023-08-11T15:47:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.752+0000 7f075d6feb80 1 -- start start 2023-08-11T15:47:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875310 0x5590a98756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875c30 0x5590a98708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9870e20 0x5590a9871280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f075d6feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a97c55d0 con 0x5590a9875310 2023-08-11T15:47:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f075d6feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a97c2ef0 con 0x5590a9875c30 2023-08-11T15:47:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f075d6feb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a97be7e0 con 0x5590a9870e20 2023-08-11T15:47:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f07437fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875c30 0x5590a98708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f07437fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875c30 0x5590a98708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:41026/0 (socket says 172.21.15.32:41026) 2023-08-11T15:47:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.753+0000 7f07437fe700 1 -- 172.21.15.32:0/1294394969 learned_addr learned my addr 172.21.15.32:0/1294394969 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f07437fe700 1 -- 172.21.15.32:0/1294394969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9870e20 msgr2=0x5590a9871280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T15:47:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f07437fe700 1 --2- 172.21.15.32:0/1294394969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9870e20 0x5590a9871280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f07437fe700 1 -- 172.21.15.32:0/1294394969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875310 msgr2=0x5590a98756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f07437fe700 1 --2- 172.21.15.32:0/1294394969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875310 0x5590a98756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f07437fe700 1 -- 172.21.15.32:0/1294394969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a963e5d0 con 0x5590a9875c30 2023-08-11T15:47:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f07437fe700 1 --2- 172.21.15.32:0/1294394969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875c30 0x5590a98708e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f07340c9360 tx=0x7f07340cfeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16c94ed199bbad0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f0740ff9700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f072c0a0260 0x7f072c0a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.754+0000 7f0729ffb700 1 -- 172.21.15.32:0/1294394969 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07340d4b50 con 0x5590a9875c30 2023-08-11T15:47:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.755+0000 7f0729ffb700 1 -- 172.21.15.32:0/1294394969 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07340d4cb0 con 0x5590a9875c30 2023-08-11T15:47:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.755+0000 7f0729ffb700 1 -- 172.21.15.32:0/1294394969 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07340d9600 con 0x5590a9875c30 2023-08-11T15:47:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.755+0000 7f0740ff9700 1 -- 172.21.15.32:0/3553340342 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f0734014070 con 0x7f0744100ad0 2023-08-11T15:47:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 -- 172.21.15.32:0/1294394969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875c30 msgr2=0x5590a98708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1294394969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875c30 0x5590a98708e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f07340c9360 tx=0x7f07340cfeb0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f0743fff700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f072c0a0260 0x7f072c0a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 -- 172.21.15.32:0/1294394969 shutdown_connections 2023-08-11T15:47:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1294394969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875310 0x5590a98756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1294394969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9870e20 0x5590a9871280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1294394969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875c30 0x5590a98708e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 -- 172.21.15.32:0/1294394969 >> 172.21.15.32:0/1294394969 conn(0x5590a97998f0 msgr2=0x5590a9866e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 -- 172.21.15.32:0/1294394969 shutdown_connections 2023-08-11T15:47:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.756+0000 7f075d6feb80 1 -- 172.21.15.32:0/1294394969 wait complete. 2023-08-11T15:47:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.757+0000 7f0743fff700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f072c0a0260 0x7f072c0a2720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x5590a9867b80 tx=0x7f073c009920 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.757+0000 7f075d6feb80 1 -- start start 2023-08-11T15:47:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.757+0000 7f075d6feb80 1 Processor -- start 2023-08-11T15:47:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.758+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.758+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a9876110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.758+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9876650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.758+0000 7f075d6feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a975ca60 con 0x5590a9870e20 2023-08-11T15:47:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.758+0000 7f075d6feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a975c760 con 0x5590a9875310 2023-08-11T15:47:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.758+0000 7f075d6feb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a96408c0 con 0x5590a9875c30 2023-08-11T15:47:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.758+0000 7f075d6feb80 10 client.0 _ll_register_callbacks cb 0x5590a975fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f07437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f07437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54180/0 (socket says 172.21.15.32:54180) 2023-08-11T15:47:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f07437fe700 1 -- 172.21.15.32:0/140108528 learned_addr learned my addr 172.21.15.32:0/140108528 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f0743fff700 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9876650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f0742ffd700 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a9876110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f07437fe700 1 -- 172.21.15.32:0/140108528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 msgr2=0x5590a9876650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f07437fe700 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9876650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f07437fe700 1 -- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 msgr2=0x5590a9876110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.759+0000 7f07437fe700 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a9876110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.760+0000 7f07437fe700 1 -- 172.21.15.32:0/140108528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07441a3640 con 0x5590a9870e20 2023-08-11T15:47:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.760+0000 7f07437fe700 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987cad0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07340957b0 tx=0x7f07340cf7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f56d0c87bfa1a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.760+0000 7f0722ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:47:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.760+0000 7f0722ffd700 1 -- 172.21.15.32:0/140108528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07340cf520 con 0x5590a9870e20 2023-08-11T15:47:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.760+0000 7f0722ffd700 1 -- 172.21.15.32:0/140108528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f0734016330 con 0x5590a9870e20 2023-08-11T15:47:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.760+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590a985f680 con 0x5590a9870e20 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.761+0000 7f0722ffd700 1 -- 172.21.15.32:0/140108528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f073400cdf0 con 0x5590a9870e20 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.761+0000 7f075d6feb80 10 client.15438 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.761+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5590a95ead80 con 0x5590a9870e20 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.761+0000 7f0722ffd700 1 -- 172.21.15.32:0/140108528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590a985f680 con 0x5590a9870e20 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f0722ffd700 1 -- 172.21.15.32:0/140108528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07340d4950 con 0x5590a9870e20 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f075d6feb80 10 client.15438 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f075d6feb80 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f075d6feb80 20 client.15438 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f075d6feb80 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a988e470 0x5590a9890860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f075d6feb80 4 client.15438 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:47:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590a983b480 con 0x5590a988e470 2023-08-11T15:47:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.762+0000 7f0742ffd700 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a988e470 0x5590a9890860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.763+0000 7f0742ffd700 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a988e470 0x5590a9890860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a59ac5e01528d0 server_cookie=e915eab997768758 in_seq=0 out_seq=0 2023-08-11T15:47:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.763+0000 7f0722ffd700 10 client.15438 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f0722ffd700 1 -- 172.21.15.32:0/140108528 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5590a983b480 con 0x5590a988e470 2023-08-11T15:47:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f0722ffd700 10 client.15438 handle_command_reply: tid=0 2023-08-11T15:47:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 client.15438 shutdown 2023-08-11T15:47:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 msgr2=0x5590a987cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987cad0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07340957b0 tx=0x7f07340cf7c0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 shutdown_connections 2023-08-11T15:47:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a988e470 0x5590a9890860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987cad0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9876650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 --2- 172.21.15.32:0/140108528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a9876110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 >> 172.21.15.32:0/140108528 conn(0x5590a97998f0 msgr2=0x5590a9861960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f0722ffd700 0 client.15438 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 shutdown_connections 2023-08-11T15:47:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.764+0000 7f075d6feb80 1 -- 172.21.15.32:0/140108528 wait complete. 2023-08-11T15:47:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.765+0000 7f075d6feb80 20 client.15438 trim_cache size 0 max 16384 2023-08-11T15:47:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 Processor -- start 2023-08-11T15:47:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 -- start start 2023-08-11T15:47:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a98744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9874a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a986ef70 con 0x5590a9870e20 2023-08-11T15:47:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a9867950 con 0x5590a9875310 2023-08-11T15:47:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.795+0000 7f075d6feb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a987a9f0 con 0x5590a9875c30 2023-08-11T15:47:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f07437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0742ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a98744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f07437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54192/0 (socket says 172.21.15.32:54192) 2023-08-11T15:47:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f07437fe700 1 -- 172.21.15.32:0/1901937401 learned_addr learned my addr 172.21.15.32:0/1901937401 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0743fff700 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9874a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0742ffd700 1 -- 172.21.15.32:0/1901937401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 msgr2=0x5590a9874a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0742ffd700 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9874a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0742ffd700 1 -- 172.21.15.32:0/1901937401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 msgr2=0x5590a987a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0742ffd700 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0742ffd700 1 -- 172.21.15.32:0/1901937401 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a9860bc0 con 0x5590a9875310 2023-08-11T15:47:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.796+0000 7f0742ffd700 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a98744c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5590a9890ff0 tx=0x7f073801bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f112211976316749 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.797+0000 7f072a7fc700 1 -- 172.21.15.32:0/1901937401 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738016040 con 0x5590a9875310 2023-08-11T15:47:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.797+0000 7f072a7fc700 1 -- 172.21.15.32:0/1901937401 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f073801f020 con 0x5590a9875310 2023-08-11T15:47:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.797+0000 7f072a7fc700 1 -- 172.21.15.32:0/1901937401 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380047c0 con 0x5590a9875310 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 -- 172.21.15.32:0/1901937401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 msgr2=0x5590a98744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a98744c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5590a9890ff0 tx=0x7f073801bb20 comp rx=0 tx=0).stop 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 -- 172.21.15.32:0/1901937401 shutdown_connections 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9870e20 0x5590a987a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875c30 0x5590a9874a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 --2- 172.21.15.32:0/1901937401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9875310 0x5590a98744c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 -- 172.21.15.32:0/1901937401 >> 172.21.15.32:0/1901937401 conn(0x5590a97998f0 msgr2=0x5590a987c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 -- 172.21.15.32:0/1901937401 shutdown_connections 2023-08-11T15:47:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 -- 172.21.15.32:0/1901937401 wait complete. 2023-08-11T15:47:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.798+0000 7f075d6feb80 1 -- start start 2023-08-11T15:47:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 1 Processor -- start 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9870e20 0x5590a988b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875310 0x5590a988b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875c30 0x5590a9885c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a9886360 con 0x5590a9875c30 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a98864d0 con 0x5590a9870e20 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a9886690 con 0x5590a9875310 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.799+0000 7f075d6feb80 10 client.0 _ll_register_callbacks cb 0x7f07340cfd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f0743fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875c30 0x5590a9885c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f0742ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875310 0x5590a988b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f07437fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9870e20 0x5590a988b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f0742ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875310 0x5590a988b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43062/0 (socket says 172.21.15.32:43062) 2023-08-11T15:47:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f0742ffd700 1 -- 172.21.15.32:0/212417497 learned_addr learned my addr 172.21.15.32:0/212417497 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f07437fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9870e20 0x5590a988b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:41064/0 (socket says 172.21.15.32:41064) 2023-08-11T15:47:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f07437fe700 1 -- 172.21.15.32:0/212417497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875310 msgr2=0x5590a988b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f07437fe700 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875310 0x5590a988b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f07437fe700 1 -- 172.21.15.32:0/212417497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875c30 msgr2=0x5590a9885c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f07437fe700 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875c30 0x5590a9885c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f07437fe700 1 -- 172.21.15.32:0/212417497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0738009b90 con 0x5590a9870e20 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.800+0000 7f0743fff700 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875c30 0x5590a9885c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f07437fe700 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9870e20 0x5590a988b0b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07340991f0 tx=0x7f073400c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10154a698b71281 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f0722ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f0722ffd700 1 -- 172.21.15.32:0/212417497 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07340cf520 con 0x5590a9870e20 2023-08-11T15:47:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07340d4cb0 con 0x5590a9870e20 2023-08-11T15:47:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f0722ffd700 1 -- 172.21.15.32:0/212417497 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f0734016330 con 0x5590a9870e20 2023-08-11T15:47:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f0722ffd700 1 -- 172.21.15.32:0/212417497 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f073400cdf0 con 0x5590a9870e20 2023-08-11T15:47:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f075d6feb80 10 client.25454 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5590a95d3c00 con 0x5590a9870e20 2023-08-11T15:47:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.801+0000 7f0722ffd700 1 -- 172.21.15.32:0/212417497 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07340d4cb0 con 0x5590a9870e20 2023-08-11T15:47:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f0722ffd700 1 -- 172.21.15.32:0/212417497 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07340d4950 con 0x5590a9870e20 2023-08-11T15:47:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f075d6feb80 10 client.25454 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f075d6feb80 10 client.25454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f075d6feb80 20 client.25454 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f075d6feb80 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a98e2c60 0x5590a98e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f075d6feb80 4 client.25454 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:47:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5590a97b77c0 con 0x5590a98e2c60 2023-08-11T15:47:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.802+0000 7f0742ffd700 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a98e2c60 0x5590a98e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.803+0000 7f0742ffd700 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a98e2c60 0x5590a98e5050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81897f40cddf1a8f server_cookie=ebf43806ad47498b in_seq=0 out_seq=0 2023-08-11T15:47:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.803+0000 7f0722ffd700 10 client.25454 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f0722ffd700 1 -- 172.21.15.32:0/212417497 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5590a97b77c0 con 0x5590a98e2c60 2023-08-11T15:47:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f0722ffd700 10 client.25454 handle_command_reply: tid=0 2023-08-11T15:47:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 client.25454 shutdown 2023-08-11T15:47:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9870e20 msgr2=0x5590a988b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9870e20 0x5590a988b0b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07340991f0 tx=0x7f073400c750 comp rx=0 tx=0).stop 2023-08-11T15:47:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 shutdown_connections 2023-08-11T15:47:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5590a98e2c60 0x5590a98e5050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5590a9875c30 0x5590a9885c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590a9870e20 0x5590a988b0b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f0722ffd700 0 client.25454 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 --2- 172.21.15.32:0/212417497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5590a9875310 0x5590a988b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 >> 172.21.15.32:0/212417497 conn(0x5590a97998f0 msgr2=0x5590a98744c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 shutdown_connections 2023-08-11T15:47:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.814+0000 7f075d6feb80 1 -- 172.21.15.32:0/212417497 wait complete. 2023-08-11T15:47:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.815+0000 7f075d6feb80 20 client.25454 trim_cache size 0 max 16384 2023-08-11T15:47:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:47:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:47:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:47:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:47:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.815+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f072c0a0260 msgr2=0x7f072c0a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.815+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f072c0a0260 0x7f072c0a2720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x5590a9867b80 tx=0x7f073c009920 comp rx=0 tx=0).stop 2023-08-11T15:47:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.815+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744100ad0 msgr2=0x7f074419c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.815+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744100ad0 0x7f074419c830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f073400ec10 tx=0x7f073400ef20 comp rx=0 tx=0).stop 2023-08-11T15:47:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 shutdown_connections 2023-08-11T15:47:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f072c0a0260 0x7f072c0a2720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0744100ad0 0x7f074419c830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0744102bd0 0x7f07441968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 --2- 172.21.15.32:0/3553340342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0744102210 0x7f074419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 >> 172.21.15.32:0/3553340342 conn(0x7f07440fc1e0 msgr2=0x7f07440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 shutdown_connections 2023-08-11T15:47:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:23.816+0000 7f0748b9d700 1 -- 172.21.15.32:0/3553340342 wait complete. 2023-08-11T15:47:24.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:24 smithi032 ceph-mon[97103]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:24 smithi192 ceph-mon[130806]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:24.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:24 smithi134 ceph-mon[130447]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:26 smithi032 ceph-mon[97103]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2023-08-11T15:47:26.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:26 smithi192 ceph-mon[130806]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2023-08-11T15:47:26.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:26 smithi134 ceph-mon[130447]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2023-08-11T15:47:28.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:27 smithi134 ceph-mon[130447]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-08-11T15:47:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:27 smithi032 ceph-mon[97103]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-08-11T15:47:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:27 smithi192 ceph-mon[130806]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-08-11T15:47:30.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:30 smithi032 ceph-mon[97103]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:30 smithi192 ceph-mon[130806]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:30 smithi134 ceph-mon[130447]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:32.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:32 smithi032 ceph-mon[97103]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-08-11T15:47:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:32 smithi192 ceph-mon[130806]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-08-11T15:47:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:32 smithi134 ceph-mon[130447]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-08-11T15:47:34.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:34 smithi032 ceph-mon[97103]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:34.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:34 smithi192 ceph-mon[130806]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:34 smithi134 ceph-mon[130447]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-08-11T15:47:36.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:36 smithi032 ceph-mon[97103]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:36.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:36 smithi192 ceph-mon[130806]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:36.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:36 smithi134 ceph-mon[130447]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 -- 172.21.15.32:0/3630870625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b4072cd0 msgr2=0x7f76b40730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/3630870625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b4072cd0 0x7f76b40730b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f76a80098f0 tx=0x7f76a8009c00 comp rx=0 tx=0).stop 2023-08-11T15:47:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 -- 172.21.15.32:0/3630870625 shutdown_connections 2023-08-11T15:47:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/3630870625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b410d1a0 0x7f76b410f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/3630870625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b4072cd0 0x7f76b40730b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/3630870625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40735f0 0x7f76b410cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 -- 172.21.15.32:0/3630870625 >> 172.21.15.32:0/3630870625 conn(0x7f76b40fc3f0 msgr2=0x7f76b40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 -- 172.21.15.32:0/3630870625 shutdown_connections 2023-08-11T15:47:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.056+0000 7f76bb1ff700 1 -- 172.21.15.32:0/3630870625 wait complete. 2023-08-11T15:47:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.058+0000 7f76bb1ff700 1 Processor -- start 2023-08-11T15:47:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.058+0000 7f76bb1ff700 1 -- start start 2023-08-11T15:47:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.058+0000 7f76bb1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b4072cd0 0x7f76b41008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.058+0000 7f76bb1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b40735f0 0x7f76b4100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.058+0000 7f76bb1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b410d1a0 0x7f76b41a9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76bb1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4101660 con 0x7f76b410d1a0 2023-08-11T15:47:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76bb1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b41a9790 con 0x7f76b4072cd0 2023-08-11T15:47:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76bb1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b41a98d0 con 0x7f76b40735f0 2023-08-11T15:47:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b40735f0 0x7f76b4100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b979c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b410d1a0 0x7f76b41a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b40735f0 0x7f76b4100e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39390/0 (socket says 172.21.15.32:39390) 2023-08-11T15:47:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b3fff700 1 -- 172.21.15.32:0/396835006 learned_addr learned my addr 172.21.15.32:0/396835006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b4072cd0 0x7f76b41008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b3fff700 1 -- 172.21.15.32:0/396835006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b4072cd0 msgr2=0x7f76b41008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b3fff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b4072cd0 0x7f76b41008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.059+0000 7f76b3fff700 1 -- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b410d1a0 msgr2=0x7f76b41a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.060+0000 7f76b3fff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b410d1a0 0x7f76b41a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.060+0000 7f76b3fff700 1 -- 172.21.15.32:0/396835006 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8009520 con 0x7f76b40735f0 2023-08-11T15:47:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.060+0000 7f76b3fff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b40735f0 0x7f76b4100e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9c0 tx=0x7f76a000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.060+0000 7f76b1ffb700 1 -- 172.21.15.32:0/396835006 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000cbd0 con 0x7f76b40735f0 2023-08-11T15:47:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.060+0000 7f76b1ffb700 1 -- 172.21.15.32:0/396835006 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76a000eeb0 con 0x7f76b40735f0 2023-08-11T15:47:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.060+0000 7f76b1ffb700 1 -- 172.21.15.32:0/396835006 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0005580 con 0x7f76b40735f0 2023-08-11T15:47:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.060+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b41a9b30 con 0x7f76b40735f0 2023-08-11T15:47:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.061+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b41a9ff0 con 0x7f76b40735f0 2023-08-11T15:47:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.067+0000 7f76b1ffb700 1 -- 172.21.15.32:0/396835006 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7f76a0004750 con 0x7f76b40735f0 2023-08-11T15:47:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.068+0000 7f76ceb95b80 1 Processor -- start 2023-08-11T15:47:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.068+0000 7f76ceb95b80 1 -- start start 2023-08-11T15:47:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d0195d0 con 0x55f18d0c5e80 2023-08-11T15:47:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d016ef0 con 0x55f18d0c0630 2023-08-11T15:47:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d0127e0 con 0x55f18d0c0ff0 2023-08-11T15:47:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76b979c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76b979c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40062/0 (socket says 172.21.15.32:40062) 2023-08-11T15:47:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76b979c700 1 -- 172.21.15.32:0/1062112060 learned_addr learned my addr 172.21.15.32:0/1062112060 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76b1ffb700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f76a40a02e0 0x7f76a40a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.069+0000 7f76b1ffb700 1 -- 172.21.15.32:0/396835006 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f76a0014070 con 0x7f76b40735f0 2023-08-11T15:47:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.070+0000 7f76b979c700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f76a40a02e0 0x7f76a40a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.070+0000 7f76b979c700 1 -- 172.21.15.32:0/1062112060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 msgr2=0x55f18d0ccc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:47:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.070+0000 7f76b979c700 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0ccc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.070+0000 7f76b979c700 1 -- 172.21.15.32:0/1062112060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 msgr2=0x55f18d0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.070+0000 7f76b979c700 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.070+0000 7f76b979c700 1 -- 172.21.15.32:0/1062112060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18ce925d0 con 0x55f18d0c5e80 2023-08-11T15:47:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.070+0000 7f76b979c700 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0c6260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00fc70 tx=0x7f76ac00ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53697e4e1821c8b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.071+0000 7f769effd700 1 -- 172.21.15.32:0/1062112060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00e040 con 0x55f18d0c5e80 2023-08-11T15:47:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.071+0000 7f76b979c700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f76a40a02e0 0x7f76a40a27a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009000 tx=0x7f76ac01e000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.071+0000 7f769effd700 1 -- 172.21.15.32:0/1062112060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76ac00f040 con 0x55f18d0c5e80 2023-08-11T15:47:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.071+0000 7f769effd700 1 -- 172.21.15.32:0/1062112060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac01a800 con 0x55f18d0c5e80 2023-08-11T15:47:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1062112060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 msgr2=0x55f18d0c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0c6260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00fc70 tx=0x7f76ac00ff80 comp rx=0 tx=0).stop 2023-08-11T15:47:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1062112060 shutdown_connections 2023-08-11T15:47:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0c6260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1062112060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d0c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1062112060 >> 172.21.15.32:0/1062112060 conn(0x55f18cfed8f0 msgr2=0x55f18d0bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1062112060 shutdown_connections 2023-08-11T15:47:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.072+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1062112060 wait complete. 2023-08-11T15:47:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.073+0000 7f76ceb95b80 1 -- start start 2023-08-11T15:47:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.073+0000 7f76ceb95b80 1 Processor -- start 2023-08-11T15:47:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.074+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0630 0x55f18d0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.074+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.074+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c5e80 0x55f18d0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.074+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18cfb0a60 con 0x55f18d0c0ff0 2023-08-11T15:47:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.074+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18cfb0760 con 0x55f18d0c5e80 2023-08-11T15:47:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.074+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18ce948c0 con 0x55f18d0c0630 2023-08-11T15:47:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.074+0000 7f76ceb95b80 10 client.0 _ll_register_callbacks cb 0x55f18cfb3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40074/0 (socket says 172.21.15.32:40074) 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 -- 172.21.15.32:0/1032212241 learned_addr learned my addr 172.21.15.32:0/1032212241 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b979c700 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c5e80 0x55f18d0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 -- 172.21.15.32:0/1032212241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0630 msgr2=0x55f18d0cfa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0630 0x55f18d0cfa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 -- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c5e80 msgr2=0x55f18d0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c5e80 0x55f18d0c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.075+0000 7f76b3fff700 1 -- 172.21.15.32:0/1032212241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac00f820 con 0x55f18d0c0ff0 2023-08-11T15:47:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.076+0000 7f76b3fff700 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cffb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f76a00990a0 tx=0x7f76a000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6606e48b0bc40 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.076+0000 7f7693fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:47:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.076+0000 7f7693fff700 1 -- 172.21.15.32:0/1032212241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a009cf40 con 0x55f18d0c0ff0 2023-08-11T15:47:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.076+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f18d0b37e0 con 0x55f18d0c0ff0 2023-08-11T15:47:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.076+0000 7f7693fff700 1 -- 172.21.15.32:0/1032212241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f76a00a0f20 con 0x55f18d0c0ff0 2023-08-11T15:47:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f7693fff700 1 -- 172.21.15.32:0/1032212241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a00b1220 con 0x55f18d0c0ff0 2023-08-11T15:47:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 10 client.15456 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f18ce3ed80 con 0x55f18d0c0ff0 2023-08-11T15:47:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f7693fff700 1 -- 172.21.15.32:0/1032212241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f18d0b37e0 con 0x55f18d0c0ff0 2023-08-11T15:47:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f7693fff700 1 -- 172.21.15.32:0/1032212241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f76a00c9af0 con 0x55f18d0c0ff0 2023-08-11T15:47:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 20 client.15456 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d0da010 0x55f18d0dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 4 client.15456 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:47:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.077+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f18d08f480 con 0x55f18d0da010 2023-08-11T15:47:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.078+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d0da010 0x55f18d0dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.079+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d0da010 0x55f18d0dc430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc902ce848b2b67 server_cookie=da8951f7f0a70a57 in_seq=0 out_seq=0 2023-08-11T15:47:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.079+0000 7f7693fff700 10 client.15456 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f7693fff700 1 -- 172.21.15.32:0/1032212241 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f18d08f480 con 0x55f18d0da010 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f7693fff700 10 client.15456 handle_command_reply: tid=0 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 client.15456 shutdown 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c0ff0 msgr2=0x55f18d0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cffb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f76a00990a0 tx=0x7f76a000cdb0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 shutdown_connections 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d0da010 0x55f18d0dc430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c5e80 0x55f18d0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0630 0x55f18d0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/1032212241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cffb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.080+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 >> 172.21.15.32:0/1032212241 conn(0x55f18cfed8f0 msgr2=0x55f18d0ba5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.081+0000 7f7693fff700 0 client.15456 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.081+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 shutdown_connections 2023-08-11T15:47:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.081+0000 7f76ceb95b80 1 -- 172.21.15.32:0/1032212241 wait complete. 2023-08-11T15:47:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.081+0000 7f76ceb95b80 20 client.15456 trim_cache size 0 max 16384 2023-08-11T15:47:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.110+0000 7f76ceb95b80 1 Processor -- start 2023-08-11T15:47:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.110+0000 7f76ceb95b80 1 -- start start 2023-08-11T15:47:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d0b3570 con 0x55f18d0c5e80 2023-08-11T15:47:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d0b3780 con 0x55f18d0c0630 2023-08-11T15:47:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d0d7670 con 0x55f18d0c0ff0 2023-08-11T15:47:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76b979c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39396/0 (socket says 172.21.15.32:39396) 2023-08-11T15:47:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.111+0000 7f76b3fff700 1 -- 172.21.15.32:0/3436795151 learned_addr learned my addr 172.21.15.32:0/3436795151 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f76b3fff700 1 -- 172.21.15.32:0/3436795151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 msgr2=0x55f18d0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f76b3fff700 1 --2- 172.21.15.32:0/3436795151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f76b3fff700 1 -- 172.21.15.32:0/3436795151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 msgr2=0x55f18d0bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f76b3fff700 1 --2- 172.21.15.32:0/3436795151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f76b3fff700 1 -- 172.21.15.32:0/3436795151 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18ce27c00 con 0x55f18d0c0ff0 2023-08-11T15:47:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f76b3fff700 1 --2- 172.21.15.32:0/3436795151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cf5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c5b0 tx=0x7f76a0099290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f00c5cefb78f5f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f769f7fe700 1 -- 172.21.15.32:0/3436795151 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0099810 con 0x55f18d0c0ff0 2023-08-11T15:47:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f769f7fe700 1 -- 172.21.15.32:0/3436795151 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76a0094f00 con 0x55f18d0c0ff0 2023-08-11T15:47:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.112+0000 7f769f7fe700 1 -- 172.21.15.32:0/3436795151 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a00a0a90 con 0x55f18d0c0ff0 2023-08-11T15:47:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.113+0000 7f76ceb95b80 1 -- 172.21.15.32:0/3436795151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 msgr2=0x55f18d0cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.113+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/3436795151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cf5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c5b0 tx=0x7f76a0099290 comp rx=0 tx=0).stop 2023-08-11T15:47:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 -- 172.21.15.32:0/3436795151 shutdown_connections 2023-08-11T15:47:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/3436795151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d0bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/3436795151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/3436795151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d0cf5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 -- 172.21.15.32:0/3436795151 >> 172.21.15.32:0/3436795151 conn(0x55f18cfed8f0 msgr2=0x55f18d0c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 -- 172.21.15.32:0/3436795151 shutdown_connections 2023-08-11T15:47:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 -- 172.21.15.32:0/3436795151 wait complete. 2023-08-11T15:47:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.114+0000 7f76ceb95b80 1 -- start start 2023-08-11T15:47:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 1 Processor -- start 2023-08-11T15:47:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d121c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d122150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d12cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d122a30 con 0x55f18d0c5e80 2023-08-11T15:47:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d12d1d0 con 0x55f18d0c0630 2023-08-11T15:47:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d12d310 con 0x55f18d0c0ff0 2023-08-11T15:47:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.115+0000 7f76ceb95b80 10 client.0 _ll_register_callbacks cb 0x7f76a0098d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.116+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d122150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.116+0000 7f76b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d122150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39410/0 (socket says 172.21.15.32:39410) 2023-08-11T15:47:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.116+0000 7f76b3fff700 1 -- 172.21.15.32:0/2411338318 learned_addr learned my addr 172.21.15.32:0/2411338318 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.116+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d121c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.116+0000 7f76b979c700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d12cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b8f9b700 1 -- 172.21.15.32:0/2411338318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 msgr2=0x55f18d122150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d122150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b8f9b700 1 -- 172.21.15.32:0/2411338318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 msgr2=0x55f18d12cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d12cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b8f9b700 1 -- 172.21.15.32:0/2411338318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f18ce3ed80 con 0x55f18d0c0630 2023-08-11T15:47:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b3fff700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d122150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:47:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b979c700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d12cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:47:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76b8f9b700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d121c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009810 tx=0x7f76a800f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede235f50503bdf7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f7693fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f7693fff700 1 -- 172.21.15.32:0/2411338318 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a801c040 con 0x55f18d0c0630 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f7693fff700 1 -- 172.21.15.32:0/2411338318 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f76a800f880 con 0x55f18d0c0630 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.117+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76ac00f040 con 0x55f18d0c0630 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.118+0000 7f7693fff700 1 -- 172.21.15.32:0/2411338318 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a8026940 con 0x55f18d0c0630 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.118+0000 7f76ceb95b80 10 client.25469 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.118+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f18ce651f0 con 0x55f18d0c0630 2023-08-11T15:47:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.118+0000 7f7693fff700 1 -- 172.21.15.32:0/2411338318 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76ac00f040 con 0x55f18d0c0630 2023-08-11T15:47:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f7693fff700 1 -- 172.21.15.32:0/2411338318 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f76a8015020 con 0x55f18d0c0630 2023-08-11T15:47:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f76ceb95b80 10 client.25469 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f76ceb95b80 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f76ceb95b80 20 client.25469 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d12ea70 0x55f18d130e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f76ceb95b80 4 client.25469 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:47:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f18d00b7c0 con 0x55f18d12ea70 2023-08-11T15:47:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.119+0000 7f76b3fff700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d12ea70 0x55f18d130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.120+0000 7f76b3fff700 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d12ea70 0x55f18d130e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709b46f00233df5 server_cookie=6278105f671705b5 in_seq=0 out_seq=0 2023-08-11T15:47:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.120+0000 7f7693fff700 10 client.25469 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f7693fff700 1 -- 172.21.15.32:0/2411338318 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f18d00b7c0 con 0x55f18d12ea70 2023-08-11T15:47:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f7693fff700 10 client.25469 handle_command_reply: tid=0 2023-08-11T15:47:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f76ceb95b80 1 client.25469 shutdown 2023-08-11T15:47:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 msgr2=0x55f18d121c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d121c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009810 tx=0x7f76a800f4e0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 shutdown_connections 2023-08-11T15:47:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f18d12ea70 0x55f18d130e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.124+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18d0c5e80 0x55f18d12cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.125+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f18d0c0630 0x55f18d121c10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.125+0000 7f76ceb95b80 1 --2- 172.21.15.32:0/2411338318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f18d0c0ff0 0x55f18d122150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.125+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 >> 172.21.15.32:0/2411338318 conn(0x55f18cfed8f0 msgr2=0x55f18d0c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.125+0000 7f7693fff700 0 client.25469 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.125+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 shutdown_connections 2023-08-11T15:47:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.125+0000 7f76ceb95b80 1 -- 172.21.15.32:0/2411338318 wait complete. 2023-08-11T15:47:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.125+0000 7f76ceb95b80 20 client.25469 trim_cache size 0 max 16384 2023-08-11T15:47:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:47:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:47:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:47:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:47:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f76a40a02e0 msgr2=0x7f76a40a27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f76a40a02e0 0x7f76a40a27a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009000 tx=0x7f76ac01e000 comp rx=0 tx=0).stop 2023-08-11T15:47:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b40735f0 msgr2=0x7f76b4100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b40735f0 0x7f76b4100e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9c0 tx=0x7f76a000ecd0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 shutdown_connections 2023-08-11T15:47:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f76a40a02e0 0x7f76a40a27a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b410d1a0 0x7f76b41a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b4072cd0 0x7f76b41008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 --2- 172.21.15.32:0/396835006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b40735f0 0x7f76b4100e00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 >> 172.21.15.32:0/396835006 conn(0x7f76b40fc3f0 msgr2=0x7f76b410f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.126+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 shutdown_connections 2023-08-11T15:47:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:37.127+0000 7f76bb1ff700 1 -- 172.21.15.32:0/396835006 wait complete. 2023-08-11T15:47:38.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:37 smithi134 ceph-mon[130447]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:37 smithi032 ceph-mon[97103]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:37 smithi192 ceph-mon[130806]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-08-11T15:47:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:40 smithi032 ceph-mon[97103]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2023-08-11T15:47:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:40 smithi192 ceph-mon[130806]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2023-08-11T15:47:40.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:40 smithi134 ceph-mon[130447]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2023-08-11T15:47:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:42 smithi032 ceph-mon[97103]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:47:42.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:42 smithi192 ceph-mon[130806]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:47:42.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:42 smithi134 ceph-mon[130447]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2023-08-11T15:47:43.377 INFO:tasks.workunit.client.0.smithi032.stderr:Updating files: 61% (7381/11920) Updating files: 62% (7391/11920) Updating files: 63% (7510/11920) Updating files: 64% (7629/11920) Updating files: 65% (7748/11920) Updating files: 66% (7868/11920) Updating files: 67% (7987/11920) Updating files: 68% (8106/11920) Updating files: 69% (8225/11920) Updating files: 70% (8344/11920) Updating files: 71% (8464/11920) Updating files: 72% (8583/11920) Updating files: 73% (8702/11920) Updating files: 74% (8821/11920) Updating files: 75% (8940/11920) Updating files: 76% (9060/11920) Updating files: 77% (9179/11920) Updating files: 78% (9298/11920) Updating files: 79% (9417/11920) Updating files: 80% (9536/11920) Updating files: 81% (9656/11920) Updating files: 82% (9775/11920) Updating files: 83% (9894/11920) Updating files: 84% (10013/11920) Updating files: 85% (10132/11920) Updating files: 86% (10252/11920) Updating files: 87% (10371/11920) Updating files: 88% (10490/11920) Updating files: 89% (10609/11920) Updating files: 90% (10728/11920) Updating files: 91% (10848/11920) Updating files: 92% (10967/11920) Updating files: 93% (11086/11920) Updating files: 94% (11205/11920) Updating files: 95% (11324/11920) Updating files: 96% (11444/11920) Updating files: 96% (11496/11920) Updating files: 97% (11563/11920) Updating files: 98% (11682/11920) Updating files: 99% (11801/11920) Updating files: 100% (11920/11920) Updating files: 100% (11920/11920), done. 2023-08-11T15:47:43.639 INFO:tasks.workunit.client.0.smithi032.stderr:Note: switching to '92684807f016ce045ac66e37fdde6f8c47857bcc'. 2023-08-11T15:47:43.639 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-08-11T15:47:43.640 INFO:tasks.workunit.client.0.smithi032.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-08-11T15:47:43.640 INFO:tasks.workunit.client.0.smithi032.stderr:changes and commit them, and you can discard any commits you make in this 2023-08-11T15:47:43.640 INFO:tasks.workunit.client.0.smithi032.stderr:state without impacting any branches by switching back to a branch. 2023-08-11T15:47:43.640 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-08-11T15:47:43.641 INFO:tasks.workunit.client.0.smithi032.stderr:If you want to create a new branch to retain commits you create, you may 2023-08-11T15:47:43.641 INFO:tasks.workunit.client.0.smithi032.stderr:do so (now or later) by using -c with the switch command. Example: 2023-08-11T15:47:43.641 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-08-11T15:47:43.641 INFO:tasks.workunit.client.0.smithi032.stderr: git switch -c 2023-08-11T15:47:43.641 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-08-11T15:47:43.641 INFO:tasks.workunit.client.0.smithi032.stderr:Or undo this operation with: 2023-08-11T15:47:43.642 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-08-11T15:47:43.642 INFO:tasks.workunit.client.0.smithi032.stderr: git switch - 2023-08-11T15:47:43.642 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-08-11T15:47:43.642 INFO:tasks.workunit.client.0.smithi032.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-08-11T15:47:43.642 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-08-11T15:47:43.642 INFO:tasks.workunit.client.0.smithi032.stderr:HEAD is now at 92684807f01 Merge remote-tracking branch 'rishabh/fs-auth-subcmd' into wip-rishabh-2023Aug1-b4 2023-08-11T15:47:43.643 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-08-11T15:47:43.716 INFO:tasks.workunit.client.0.smithi032.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-08-11T15:47:43.721 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-08-11T15:47:43.721 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-08-11T15:47:44.279 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-08-11T15:47:44.360 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-08-11T15:47:44.425 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-08-11T15:47:44.429 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-08-11T15:47:44.430 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-08-11T15:47:44.493 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-08-11T15:47:44.501 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:47:44.501 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-08-11T15:47:44.544 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-08-11T15:47:44.546 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-08-11T15:47:44.546 DEBUG:teuthology.orchestra.run.smithi032:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92684807f016ce045ac66e37fdde6f8c47857bcc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-08-11T15:47:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:44 smithi032 ceph-mon[97103]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:47:44.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:44 smithi192 ceph-mon[130806]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:47:44.610 INFO:tasks.workunit.client.0.smithi032.stderr:+ dbench 1 2023-08-11T15:47:44.620 INFO:tasks.workunit.client.0.smithi032.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-08-11T15:47:44.621 INFO:tasks.workunit.client.0.smithi032.stdout: 2023-08-11T15:47:44.621 INFO:tasks.workunit.client.0.smithi032.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-08-11T15:47:44.631 INFO:tasks.workunit.client.0.smithi032.stdout:0 of 1 processes prepared for launch 0 sec 2023-08-11T15:47:44.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:44 smithi134 ceph-mon[130447]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:47:44.731 INFO:tasks.workunit.client.0.smithi032.stdout:1 of 1 processes prepared for launch 0 sec 2023-08-11T15:47:44.732 INFO:tasks.workunit.client.0.smithi032.stdout:releasing clients 2023-08-11T15:47:45.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 3169 34.11 MB/sec warmup 1 sec latency 6.298 ms 2023-08-11T15:47:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:46 smithi032 ceph-mon[97103]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:47:46.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:46 smithi192 ceph-mon[130806]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:47:46.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:46 smithi134 ceph-mon[130447]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-08-11T15:47:46.731 INFO:tasks.workunit.client.0.smithi032.stdout: 1 5948 23.54 MB/sec warmup 2 sec latency 6.143 ms 2023-08-11T15:47:47.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 8352 20.44 MB/sec warmup 3 sec latency 9.799 ms 2023-08-11T15:47:48.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:47 smithi134 ceph-mon[130447]: pgmap v36: 129 pgs: 129 active+clean; 2.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 102 KiB/s wr, 6 op/s 2023-08-11T15:47:48.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:47 smithi032 ceph-mon[97103]: pgmap v36: 129 pgs: 129 active+clean; 2.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 102 KiB/s wr, 6 op/s 2023-08-11T15:47:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:47 smithi192 ceph-mon[130806]: pgmap v36: 129 pgs: 129 active+clean; 2.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 102 KiB/s wr, 6 op/s 2023-08-11T15:47:48.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 11248 19.58 MB/sec warmup 4 sec latency 25.048 ms 2023-08-11T15:47:49.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 14482 19.42 MB/sec warmup 5 sec latency 13.177 ms 2023-08-11T15:47:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 -- 172.21.15.32:0/1145390673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 msgr2=0x7f3558106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 --2- 172.21.15.32:0/1145390673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 0x7f3558106cb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35540099c0 tx=0x7f3554009cd0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 -- 172.21.15.32:0/1145390673 shutdown_connections 2023-08-11T15:47:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 --2- 172.21.15.32:0/1145390673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 0x7f355810d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 --2- 172.21.15.32:0/1145390673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 0x7f3558106cb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 --2- 172.21.15.32:0/1145390673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558067d10 0x7f3558068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 -- 172.21.15.32:0/1145390673 >> 172.21.15.32:0/1145390673 conn(0x7f3558075250 msgr2=0x7f3558075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 -- 172.21.15.32:0/1145390673 shutdown_connections 2023-08-11T15:47:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.292+0000 7f355ffff700 1 -- 172.21.15.32:0/1145390673 wait complete. 2023-08-11T15:47:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.293+0000 7f355ffff700 1 Processor -- start 2023-08-11T15:47:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.293+0000 7f355ffff700 1 -- start start 2023-08-11T15:47:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558067d10 0x7f355819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 0x7f355819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 0x7f3558196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558196eb0 con 0x7f3558067d10 2023-08-11T15:47:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355ffff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558197020 con 0x7f35581068d0 2023-08-11T15:47:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355ffff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35581971e0 con 0x7f35580686d0 2023-08-11T15:47:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355d59a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 0x7f355819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 0x7f3558196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355d59a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 0x7f355819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39924/0 (socket says 172.21.15.32:39924) 2023-08-11T15:47:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355d59a700 1 -- 172.21.15.32:0/1526652680 learned_addr learned my addr 172.21.15.32:0/1526652680 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.294+0000 7f355dd9b700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558067d10 0x7f355819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355d59a700 1 -- 172.21.15.32:0/1526652680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 msgr2=0x7f3558196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355d59a700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 0x7f3558196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355d59a700 1 -- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558067d10 msgr2=0x7f355819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355d59a700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558067d10 0x7f355819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355d59a700 1 -- 172.21.15.32:0/1526652680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35540095f0 con 0x7f35580686d0 2023-08-11T15:47:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355e59c700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 0x7f3558196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:47:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355d59a700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 0x7f355819cf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f354800e9a0 tx=0x7f354800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f354effd700 1 -- 172.21.15.32:0/1526652680 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548009790 con 0x7f35580686d0 2023-08-11T15:47:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35581974c0 con 0x7f35580686d0 2023-08-11T15:47:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f354effd700 1 -- 172.21.15.32:0/1526652680 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3548010430 con 0x7f35580686d0 2023-08-11T15:47:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f354effd700 1 -- 172.21.15.32:0/1526652680 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548018680 con 0x7f35580686d0 2023-08-11T15:47:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.295+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35581a39d0 con 0x7f35580686d0 2023-08-11T15:47:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.300+0000 7f354effd700 1 -- 172.21.15.32:0/1526652680 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7f35480105a0 con 0x7f35580686d0 2023-08-11T15:47:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.301+0000 7f35738a1b80 1 Processor -- start 2023-08-11T15:47:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.301+0000 7f35738a1b80 1 -- start start 2023-08-11T15:47:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1fe80 0x55d414a20260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1a630 0x55d414a1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f354effd700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f35440a0260 0x7f35440a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f35738a1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4149735d0 con 0x55d414a1a630 2023-08-11T15:47:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f35738a1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414970ef0 con 0x55d414a1fe80 2023-08-11T15:47:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f35738a1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41496c7e0 con 0x55d414a1aff0 2023-08-11T15:47:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1fe80 0x55d414a20260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f354effd700 1 -- 172.21.15.32:0/1526652680 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3548014070 con 0x7f35580686d0 2023-08-11T15:47:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1fe80 0x55d414a20260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:37182/0 (socket says 172.21.15.32:37182) 2023-08-11T15:47:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 -- 172.21.15.32:0/1465846229 learned_addr learned my addr 172.21.15.32:0/1465846229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355dd9b700 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1a630 0x55d414a1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355d59a700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f35440a0260 0x7f35440a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 -- 172.21.15.32:0/1465846229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 msgr2=0x55d414a26c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a26c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 -- 172.21.15.32:0/1465846229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1a630 msgr2=0x55d414a1aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.302+0000 7f355e59c700 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1a630 0x55d414a1aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.303+0000 7f355e59c700 1 -- 172.21.15.32:0/1465846229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4147ec5d0 con 0x55d414a1fe80 2023-08-11T15:47:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.303+0000 7f355e59c700 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1fe80 0x55d414a20260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3550004350 tx=0x7f3550009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7e7c3520216300 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.303+0000 7f355d59a700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f35440a0260 0x7f35440a2720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35480c9070 tx=0x7f354800c480 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.303+0000 7f3543fff700 1 -- 172.21.15.32:0/1465846229 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550009310 con 0x55d414a1fe80 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.303+0000 7f3543fff700 1 -- 172.21.15.32:0/1465846229 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f355000f040 con 0x55d414a1fe80 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.303+0000 7f3543fff700 1 -- 172.21.15.32:0/1465846229 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550007a80 con 0x55d414a1fe80 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 -- 172.21.15.32:0/1465846229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1fe80 msgr2=0x55d414a20260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1fe80 0x55d414a20260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3550004350 tx=0x7f3550009920 comp rx=0 tx=0).stop 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 -- 172.21.15.32:0/1465846229 shutdown_connections 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a26c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1fe80 0x55d414a20260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1465846229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1a630 0x55d414a1aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 -- 172.21.15.32:0/1465846229 >> 172.21.15.32:0/1465846229 conn(0x55d4149478f0 msgr2=0x55d414a14e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 -- 172.21.15.32:0/1465846229 shutdown_connections 2023-08-11T15:47:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.304+0000 7f35738a1b80 1 -- 172.21.15.32:0/1465846229 wait complete. 2023-08-11T15:47:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.305+0000 7f35738a1b80 1 -- start start 2023-08-11T15:47:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.305+0000 7f35738a1b80 1 Processor -- start 2023-08-11T15:47:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a29b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1fe80 0x55d414a23e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f35738a1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41490aa60 con 0x55d414a1fe80 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f35738a1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41490a760 con 0x55d414a1a630 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f35738a1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4147ee8c0 con 0x55d414a1aff0 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f35738a1b80 10 client.0 _ll_register_callbacks cb 0x55d41490da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a29b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a2a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39928/0 (socket says 172.21.15.32:39928) 2023-08-11T15:47:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 -- 172.21.15.32:0/3903780873 learned_addr learned my addr 172.21.15.32:0/3903780873 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355e59c700 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1fe80 0x55d414a23e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 -- 172.21.15.32:0/3903780873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 msgr2=0x55d414a29b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a29b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 -- 172.21.15.32:0/3903780873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1fe80 msgr2=0x55d414a23e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1fe80 0x55d414a23e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.306+0000 7f355dd9b700 1 -- 172.21.15.32:0/3903780873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3550004040 con 0x55d414a1aff0 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f355dd9b700 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a2a040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35540099c0 tx=0x7f355400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ab7d94dad80086 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f35417fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f35417fa700 1 -- 172.21.15.32:0/3903780873 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355401d040 con 0x55d414a1aff0 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d414a0d7e0 con 0x55d414a1aff0 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f35417fa700 1 -- 172.21.15.32:0/3903780873 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f35540055a0 con 0x55d414a1aff0 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f35417fa700 1 -- 172.21.15.32:0/3903780873 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3554003820 con 0x55d414a1aff0 2023-08-11T15:47:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f35738a1b80 10 client.25480 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.307+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55d414798d80 con 0x55d414a1aff0 2023-08-11T15:47:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35417fa700 1 -- 172.21.15.32:0/3903780873 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d414a0d7e0 con 0x55d414a1aff0 2023-08-11T15:47:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35417fa700 1 -- 172.21.15.32:0/3903780873 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f355400f880 con 0x55d414a1aff0 2023-08-11T15:47:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35738a1b80 10 client.25480 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35738a1b80 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35738a1b80 20 client.25480 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35738a1b80 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a34080 0x55d414a364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35738a1b80 4 client.25480 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:47:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4149e9480 con 0x55d414a34080 2023-08-11T15:47:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.308+0000 7f355e59c700 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a34080 0x55d414a364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.309+0000 7f355e59c700 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a34080 0x55d414a364a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a8301c333d8a24 server_cookie=91cd8bd453fa8a07 in_seq=0 out_seq=0 2023-08-11T15:47:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.309+0000 7f35417fa700 10 client.25480 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35417fa700 1 -- 172.21.15.32:0/3903780873 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d4149e9480 con 0x55d414a34080 2023-08-11T15:47:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35417fa700 10 client.25480 handle_command_reply: tid=0 2023-08-11T15:47:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 client.25480 shutdown 2023-08-11T15:47:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 msgr2=0x55d414a2a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a2a040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35540099c0 tx=0x7f355400bed0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 shutdown_connections 2023-08-11T15:47:50.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a34080 0x55d414a364a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1fe80 0x55d414a23e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a29b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 --2- 172.21.15.32:0/3903780873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1aff0 0x55d414a2a040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35417fa700 0 client.25480 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 >> 172.21.15.32:0/3903780873 conn(0x55d4149478f0 msgr2=0x55d414a145d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 shutdown_connections 2023-08-11T15:47:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.310+0000 7f35738a1b80 1 -- 172.21.15.32:0/3903780873 wait complete. 2023-08-11T15:47:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.311+0000 7f35738a1b80 20 client.25480 trim_cache size 0 max 16384 2023-08-11T15:47:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.326+0000 7f35738a1b80 1 Processor -- start 2023-08-11T15:47:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.326+0000 7f35738a1b80 1 -- start start 2023-08-11T15:47:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a14d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f35738a1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414a16df0 con 0x55d414a1aff0 2023-08-11T15:47:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f35738a1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414a0fbd0 con 0x55d414a1a630 2023-08-11T15:47:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f35738a1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414a1eba0 con 0x55d414a1fe80 2023-08-11T15:47:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a233d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:37192/0 (socket says 172.21.15.32:37192) 2023-08-11T15:47:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 -- 172.21.15.32:0/2790766239 learned_addr learned my addr 172.21.15.32:0/2790766239 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355dd9b700 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a14d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 -- 172.21.15.32:0/2790766239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 msgr2=0x55d414a298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 -- 172.21.15.32:0/2790766239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 msgr2=0x55d414a14d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a14d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.327+0000 7f355e59c700 1 -- 172.21.15.32:0/2790766239 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d414781c00 con 0x55d414a1a630 2023-08-11T15:47:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.328+0000 7f355e59c700 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a233d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3550004940 tx=0x7f3550003b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba5712f86bf5a6c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.328+0000 7f354cff9700 1 -- 172.21.15.32:0/2790766239 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355000e7c0 con 0x55d414a1a630 2023-08-11T15:47:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.328+0000 7f354cff9700 1 -- 172.21.15.32:0/2790766239 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f355000ede0 con 0x55d414a1a630 2023-08-11T15:47:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.328+0000 7f354cff9700 1 -- 172.21.15.32:0/2790766239 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550029d20 con 0x55d414a1a630 2023-08-11T15:47:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 -- 172.21.15.32:0/2790766239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 msgr2=0x55d414a233d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a233d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3550004940 tx=0x7f3550003b10 comp rx=0 tx=0).stop 2023-08-11T15:47:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 -- 172.21.15.32:0/2790766239 shutdown_connections 2023-08-11T15:47:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a233d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 --2- 172.21.15.32:0/2790766239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a14d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 -- 172.21.15.32:0/2790766239 >> 172.21.15.32:0/2790766239 conn(0x55d4149478f0 msgr2=0x55d414a247f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 -- 172.21.15.32:0/2790766239 shutdown_connections 2023-08-11T15:47:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.329+0000 7f35738a1b80 1 -- 172.21.15.32:0/2790766239 wait complete. 2023-08-11T15:47:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.330+0000 7f35738a1b80 1 -- start start 2023-08-11T15:47:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.330+0000 7f35738a1b80 1 Processor -- start 2023-08-11T15:47:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a2fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a30380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f35738a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a2e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f35738a1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414a2e8d0 con 0x55d414a1aff0 2023-08-11T15:47:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f35738a1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414a2ea40 con 0x55d414a1a630 2023-08-11T15:47:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f35738a1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414a2ec00 con 0x55d414a1fe80 2023-08-11T15:47:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f35738a1b80 10 client.0 _ll_register_callbacks cb 0x7f355400ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:47:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a30380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a30380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48094/0 (socket says 172.21.15.32:48094) 2023-08-11T15:47:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 -- 172.21.15.32:0/1459012121 learned_addr learned my addr 172.21.15.32:0/1459012121 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:47:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355e59c700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a2fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355e59c700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a2e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 -- 172.21.15.32:0/1459012121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 msgr2=0x55d414a2e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a2e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 -- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 msgr2=0x55d414a2fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:47:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a2fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355dd9b700 1 -- 172.21.15.32:0/1459012121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3550032040 con 0x55d414a1aff0 2023-08-11T15:47:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.331+0000 7f355e59c700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:47:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f355dd9b700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a30380 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3554001eb0 tx=0x7f355400bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d97dede44f3ae6b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:47:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f35417fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:47:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f35417fa700 1 -- 172.21.15.32:0/1459012121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355401d040 con 0x55d414a1aff0 2023-08-11T15:47:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f35417fa700 1 -- 172.21.15.32:0/1459012121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3554000e40 con 0x55d414a1aff0 2023-08-11T15:47:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f355000f040 con 0x55d414a1aff0 2023-08-11T15:47:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f35417fa700 1 -- 172.21.15.32:0/1459012121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3554009290 con 0x55d414a1aff0 2023-08-11T15:47:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f35738a1b80 10 client.15480 fetch_fsmap learned FSMap version 21 2023-08-11T15:47:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.332+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55d4147bf1f0 con 0x55d414a1aff0 2023-08-11T15:47:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35417fa700 1 -- 172.21.15.32:0/1459012121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f355000f040 con 0x55d414a1aff0 2023-08-11T15:47:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35417fa700 1 -- 172.21.15.32:0/1459012121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 21) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f355400b260 con 0x55d414a1aff0 2023-08-11T15:47:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35738a1b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 21 2023-08-11T15:47:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35738a1b80 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:47:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35738a1b80 20 client.15480 populate_metadata read hostname 'smithi032' 2023-08-11T15:47:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a912a0 0x55d414a936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:47:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35738a1b80 4 client.15480 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:47:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4149657c0 con 0x55d414a912a0 2023-08-11T15:47:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.333+0000 7f355e59c700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a912a0 0x55d414a936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:47:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.334+0000 7f355e59c700 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a912a0 0x55d414a936c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3728f9cde9e542d server_cookie=56c40bb966acafb8 in_seq=0 out_seq=0 2023-08-11T15:47:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.334+0000 7f35417fa700 10 client.15480 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:50.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.524+0000 7f35417fa700 1 -- 172.21.15.32:0/1459012121 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4149657c0 con 0x55d414a912a0 2023-08-11T15:47:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.524+0000 7f35417fa700 10 client.15480 handle_command_reply: tid=0 2023-08-11T15:47:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.524+0000 7f35738a1b80 1 client.15480 shutdown 2023-08-11T15:47:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 msgr2=0x55d414a30380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a30380 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3554001eb0 tx=0x7f355400bfb0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 shutdown_connections 2023-08-11T15:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d414a912a0 0x55d414a936c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d414a1fe80 0x55d414a2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d414a1a630 0x55d414a2fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 --2- 172.21.15.32:0/1459012121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d414a1aff0 0x55d414a30380 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35417fa700 0 client.15480 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:47:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 >> 172.21.15.32:0/1459012121 conn(0x55d4149478f0 msgr2=0x55d414a29860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 shutdown_connections 2023-08-11T15:47:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 1 -- 172.21.15.32:0/1459012121 wait complete. 2023-08-11T15:47:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.525+0000 7f35738a1b80 20 client.15480 trim_cache size 0 max 16384 2023-08-11T15:47:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f35440a0260 msgr2=0x7f35440a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f35440a0260 0x7f35440a2720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35480c9070 tx=0x7f354800c480 comp rx=0 tx=0).stop 2023-08-11T15:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 msgr2=0x7f355819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 0x7f355819cf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f354800e9a0 tx=0x7f354800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 shutdown_connections 2023-08-11T15:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f35440a0260 0x7f35440a2720 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558067d10 0x7f355819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35581068d0 0x7f3558196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 --2- 172.21.15.32:0/1526652680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35580686d0 0x7f355819cf20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 >> 172.21.15.32:0/1526652680 conn(0x7f3558075250 msgr2=0x7f35580fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 shutdown_connections 2023-08-11T15:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:47:50.526+0000 7f355ffff700 1 -- 172.21.15.32:0/1526652680 wait complete. 2023-08-11T15:47:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-mon[97103]: pgmap v37: 129 pgs: 129 active+clean; 4.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 334 KiB/s wr, 44 op/s 2023-08-11T15:47:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:47:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:50.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:50 smithi192 ceph-mon[130806]: pgmap v37: 129 pgs: 129 active+clean; 4.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 334 KiB/s wr, 44 op/s 2023-08-11T15:47:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:50 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:47:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:50 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:50 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:50.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:50 smithi134 ceph-mon[130447]: pgmap v37: 129 pgs: 129 active+clean; 4.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 334 KiB/s wr, 44 op/s 2023-08-11T15:47:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:50 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:47:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:50 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:50 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:50.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 16693 17.53 MB/sec warmup 6 sec latency 75.827 ms 2023-08-11T15:47:51.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setuser ceph since I am not root 2023-08-11T15:47:51.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setgroup ceph since I am not root 2023-08-11T15:47:51.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-mds[131163]: -- 172.21.15.32:0/3295372590 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591464591e0 con 0x55914642ec00 2023-08-11T15:47:51.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:47:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: starting mds.a at 2023-08-11T15:47:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:51 smithi032 ceph-mon[97103]: daemon mds.a restarted 2023-08-11T15:47:51.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:51 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:47:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:51 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:47:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:51 smithi032 ceph-mon[97103]: osdmap e94: 12 total, 12 up, 12 in 2023-08-11T15:47:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:51 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:51.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:51 smithi192 ceph-mon[130806]: daemon mds.a restarted 2023-08-11T15:47:51.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:51 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:47:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:51 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:47:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:51 smithi192 ceph-mon[130806]: osdmap e94: 12 total, 12 up, 12 in 2023-08-11T15:47:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:51 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:51.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:51 smithi134 ceph-mon[130447]: daemon mds.a restarted 2023-08-11T15:47:51.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:51 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:47:51.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:51 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:47:51.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:51 smithi134 ceph-mon[130447]: osdmap e94: 12 total, 12 up, 12 in 2023-08-11T15:47:51.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:51 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:47:51.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 19001 16.95 MB/sec warmup 7 sec latency 21.177 ms 2023-08-11T15:47:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:52 smithi032 ceph-mon[97103]: pgmap v39: 129 pgs: 129 active+clean; 4.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 401 KiB/s wr, 51 op/s 2023-08-11T15:47:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:52 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6840/2307724364,v1:172.21.15.32:6841/2307724364] up:boot 2023-08-11T15:47:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:52 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:47:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:47:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:52 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:47:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:52 smithi192 ceph-mon[130806]: pgmap v39: 129 pgs: 129 active+clean; 4.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 401 KiB/s wr, 51 op/s 2023-08-11T15:47:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:52 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6840/2307724364,v1:172.21.15.32:6841/2307724364] up:boot 2023-08-11T15:47:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:52 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:47:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:47:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:52 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:47:52.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:52 smithi134 ceph-mon[130447]: pgmap v39: 129 pgs: 129 active+clean; 4.7 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 401 KiB/s wr, 51 op/s 2023-08-11T15:47:52.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:52 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6840/2307724364,v1:172.21.15.32:6841/2307724364] up:boot 2023-08-11T15:47:52.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:52 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:47:52.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:47:52.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:52 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:47:52.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 22046 17.00 MB/sec warmup 8 sec latency 5.972 ms 2023-08-11T15:47:53.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 25107 17.17 MB/sec warmup 9 sec latency 129.055 ms 2023-08-11T15:47:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:54 smithi032 ceph-mon[97103]: pgmap v40: 129 pgs: 129 active+clean; 7.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 315 op/s 2023-08-11T15:47:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:54 smithi192 ceph-mon[130806]: pgmap v40: 129 pgs: 129 active+clean; 7.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 315 op/s 2023-08-11T15:47:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:54 smithi134 ceph-mon[130447]: pgmap v40: 129 pgs: 129 active+clean; 7.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 315 op/s 2023-08-11T15:47:54.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 27270 16.19 MB/sec warmup 10 sec latency 157.365 ms 2023-08-11T15:47:55.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 29599 16.00 MB/sec warmup 11 sec latency 50.002 ms 2023-08-11T15:47:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:56 smithi032 ceph-mon[97103]: pgmap v41: 129 pgs: 129 active+clean; 7.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 315 op/s 2023-08-11T15:47:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:56 smithi192 ceph-mon[130806]: pgmap v41: 129 pgs: 129 active+clean; 7.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 315 op/s 2023-08-11T15:47:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:56 smithi134 ceph-mon[130447]: pgmap v41: 129 pgs: 129 active+clean; 7.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 315 op/s 2023-08-11T15:47:56.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 32641 16.11 MB/sec warmup 12 sec latency 8.819 ms 2023-08-11T15:47:57.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 35774 16.30 MB/sec warmup 13 sec latency 21.205 ms 2023-08-11T15:47:58.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:47:57 smithi134 ceph-mon[130447]: pgmap v42: 129 pgs: 129 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.7 MiB/s wr, 401 op/s 2023-08-11T15:47:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:47:57 smithi032 ceph-mon[97103]: pgmap v42: 129 pgs: 129 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.7 MiB/s wr, 401 op/s 2023-08-11T15:47:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:47:57 smithi192 ceph-mon[130806]: pgmap v42: 129 pgs: 129 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.7 MiB/s wr, 401 op/s 2023-08-11T15:47:58.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 38865 16.19 MB/sec warmup 14 sec latency 10.786 ms 2023-08-11T15:47:59.732 INFO:tasks.workunit.client.0.smithi032.stdout: 1 41198 15.80 MB/sec warmup 15 sec latency 85.961 ms 2023-08-11T15:48:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:00 smithi032 ceph-mon[97103]: pgmap v43: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.6 MiB/s wr, 362 op/s 2023-08-11T15:48:00.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:00 smithi192 ceph-mon[130806]: pgmap v43: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.6 MiB/s wr, 362 op/s 2023-08-11T15:48:00.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:00 smithi134 ceph-mon[130447]: pgmap v43: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.6 MiB/s wr, 362 op/s 2023-08-11T15:48:00.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 43690 15.78 MB/sec warmup 16 sec latency 5.996 ms 2023-08-11T15:48:01.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 45096 15.19 MB/sec warmup 17 sec latency 52.263 ms 2023-08-11T15:48:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:02 smithi032 ceph-mon[97103]: pgmap v44: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 351 op/s 2023-08-11T15:48:02.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:02 smithi192 ceph-mon[130806]: pgmap v44: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 351 op/s 2023-08-11T15:48:02.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:02 smithi134 ceph-mon[130447]: pgmap v44: 129 pgs: 129 active+clean; 13 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 351 op/s 2023-08-11T15:48:02.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 47283 15.09 MB/sec warmup 18 sec latency 31.045 ms 2023-08-11T15:48:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 -- 172.21.15.32:0/1414395814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 msgr2=0x7fb19010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 --2- 172.21.15.32:0/1414395814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 0x7fb19010d780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb18800b0a0 tx=0x7fb18800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 -- 172.21.15.32:0/1414395814 shutdown_connections 2023-08-11T15:48:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 --2- 172.21.15.32:0/1414395814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1901068c0 0x7fb190106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 --2- 172.21.15.32:0/1414395814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 0x7fb19010d780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 --2- 172.21.15.32:0/1414395814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901008b0 0x7fb190100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 -- 172.21.15.32:0/1414395814 >> 172.21.15.32:0/1414395814 conn(0x7fb1900fc380 msgr2=0x7fb1900fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 -- 172.21.15.32:0/1414395814 shutdown_connections 2023-08-11T15:48:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.686+0000 7fb196bff700 1 -- 172.21.15.32:0/1414395814 wait complete. 2023-08-11T15:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.687+0000 7fb196bff700 1 Processor -- start 2023-08-11T15:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb196bff700 1 -- start start 2023-08-11T15:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb196bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1901008b0 0x7fb19019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb196bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 0x7fb19019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb196bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901068c0 0x7fb190198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb196bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190198fb0 con 0x7fb1901008b0 2023-08-11T15:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb196bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190199120 con 0x7fb1901068c0 2023-08-11T15:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb196bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901992e0 con 0x7fb190101270 2023-08-11T15:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb19519c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901068c0 0x7fb190198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb19499b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1901008b0 0x7fb19019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb19519c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901068c0 0x7fb190198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:51622/0 (socket says 172.21.15.32:51622) 2023-08-11T15:48:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb19519c700 1 -- 172.21.15.32:0/462018990 learned_addr learned my addr 172.21.15.32:0/462018990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.688+0000 7fb18ffff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 0x7fb19019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb19519c700 1 -- 172.21.15.32:0/462018990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 msgr2=0x7fb19019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb19519c700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 0x7fb19019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb19519c700 1 -- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1901008b0 msgr2=0x7fb19019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb19519c700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1901008b0 0x7fb19019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb19519c700 1 -- 172.21.15.32:0/462018990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb188009d00 con 0x7fb1901068c0 2023-08-11T15:48:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb18ffff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 0x7fb19019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb19519c700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901068c0 0x7fb190198940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb188009850 tx=0x7fb1880077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb18dffb700 1 -- 172.21.15.32:0/462018990 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18800e040 con 0x7fb1901068c0 2023-08-11T15:48:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb190199560 con 0x7fb1901068c0 2023-08-11T15:48:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb18dffb700 1 -- 172.21.15.32:0/462018990 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1880079e0 con 0x7fb1901068c0 2023-08-11T15:48:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.689+0000 7fb18dffb700 1 -- 172.21.15.32:0/462018990 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880052a0 con 0x7fb1901068c0 2023-08-11T15:48:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.690+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1901a5ad0 con 0x7fb1901068c0 2023-08-11T15:48:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.693+0000 7fb18dffb700 1 -- 172.21.15.32:0/462018990 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) v1 ==== 134904+0+0 (secure 0 0 0) 0x7fb18801d470 con 0x7fb1901068c0 2023-08-11T15:48:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.694+0000 7fb1aa538b80 1 Processor -- start 2023-08-11T15:48:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.694+0000 7fb1aa538b80 1 -- start start 2023-08-11T15:48:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.694+0000 7fb18dffb700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb1800a01e0 0x7fb1800a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.694+0000 7fb18dffb700 1 -- 172.21.15.32:0/462018990 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb1880cce80 con 0x7fb1901068c0 2023-08-11T15:48:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.694+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfb2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.694+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfacab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfb8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebf055d0 con 0x55c6ebfacff0 2023-08-11T15:48:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebf02ef0 con 0x55c6ebfb1e80 2023-08-11T15:48:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebefe7e0 con 0x55c6ebfac630 2023-08-11T15:48:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb18ffff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb1800a01e0 0x7fb1800a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfb2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfb2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:51634/0 (socket says 172.21.15.32:51634) 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 -- 172.21.15.32:0/4042788792 learned_addr learned my addr 172.21.15.32:0/4042788792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb18ffff700 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfacab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19519c700 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfb8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 -- 172.21.15.32:0/4042788792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 msgr2=0x55c6ebfacab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfacab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 -- 172.21.15.32:0/4042788792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 msgr2=0x55c6ebfb8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfb8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19499b700 1 -- 172.21.15.32:0/4042788792 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ebd7e5d0 con 0x55c6ebfb1e80 2023-08-11T15:48:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.695+0000 7fb19519c700 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfb8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:48:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.696+0000 7fb19499b700 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfb2260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb184009d00 tx=0x7fb184011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f1d9f88276fdd9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.696+0000 7fb18ffff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb1800a01e0 0x7fb1800a26a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55c6ebfa7e90 tx=0x7fb17c009cc0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.696+0000 7fb17affd700 1 -- 172.21.15.32:0/4042788792 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184004230 con 0x55c6ebfb1e80 2023-08-11T15:48:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.696+0000 7fb17affd700 1 -- 172.21.15.32:0/4042788792 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb184004390 con 0x55c6ebfb1e80 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.696+0000 7fb17affd700 1 -- 172.21.15.32:0/4042788792 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184019880 con 0x55c6ebfb1e80 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.696+0000 7fb1aa538b80 1 -- 172.21.15.32:0/4042788792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfb1e80 msgr2=0x55c6ebfb2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.696+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfb2260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb184009d00 tx=0x7fb184011720 comp rx=0 tx=0).stop 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 -- 172.21.15.32:0/4042788792 shutdown_connections 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfb8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfb2260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/4042788792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfacab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 -- 172.21.15.32:0/4042788792 >> 172.21.15.32:0/4042788792 conn(0x55c6ebed98f0 msgr2=0x55c6ebfa6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 -- 172.21.15.32:0/4042788792 shutdown_connections 2023-08-11T15:48:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 -- 172.21.15.32:0/4042788792 wait complete. 2023-08-11T15:48:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.697+0000 7fb1aa538b80 1 -- start start 2023-08-11T15:48:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 1 Processor -- start 2023-08-11T15:48:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfc4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebe9ca60 con 0x55c6ebfb1e80 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebe9c760 con 0x55c6ebfacff0 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebd808c0 con 0x55c6ebfac630 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.698+0000 7fb1aa538b80 10 client.0 _ll_register_callbacks cb 0x55c6ebe9fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfc4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfc4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34554/0 (socket says 172.21.15.32:34554) 2023-08-11T15:48:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 -- 172.21.15.32:0/1553077106 learned_addr learned my addr 172.21.15.32:0/1553077106 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:51650/0 (socket says 172.21.15.32:51650) 2023-08-11T15:48:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19519c700 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 -- 172.21.15.32:0/1553077106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 msgr2=0x55c6ebfc46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 -- 172.21.15.32:0/1553077106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfb1e80 msgr2=0x55c6ebfbe530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 -- 172.21.15.32:0/1553077106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb184009960 con 0x55c6ebfac630 2023-08-11T15:48:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19519c700 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:48:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb19499b700 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfc4190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb184000c00 tx=0x7fb1840038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1342e4abcc5a7321 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.699+0000 7fb16ffff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:48:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb16ffff700 1 -- 172.21.15.32:0/1553077106 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184005ce0 con 0x55c6ebfac630 2023-08-11T15:48:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6ebf9f7e0 con 0x55c6ebfac630 2023-08-11T15:48:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb16ffff700 1 -- 172.21.15.32:0/1553077106 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb184005250 con 0x55c6ebfac630 2023-08-11T15:48:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb16ffff700 1 -- 172.21.15.32:0/1553077106 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb184005430 con 0x55c6ebfac630 2023-08-11T15:48:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb1aa538b80 10 client.25504 fetch_fsmap learned FSMap version 23 2023-08-11T15:48:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c6ebd2ad80 con 0x55c6ebfac630 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb16ffff700 1 -- 172.21.15.32:0/1553077106 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6ebf9f7e0 con 0x55c6ebfac630 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.700+0000 7fb16ffff700 1 -- 172.21.15.32:0/1553077106 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb184028020 con 0x55c6ebfac630 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb1aa538b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 23 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb1aa538b80 10 client.25504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb1aa538b80 20 client.25504 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ebfce680 0x55c6ebfd0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb1aa538b80 4 client.25504 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:48:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6ebf7b480 con 0x55c6ebfce680 2023-08-11T15:48:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb18ffff700 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ebfce680 0x55c6ebfd0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb18ffff700 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ebfce680 0x55c6ebfd0aa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31107c611e6a4caa server_cookie=f8a7f6285387b267 in_seq=0 out_seq=0 2023-08-11T15:48:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.701+0000 7fb16ffff700 10 client.25504 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.702+0000 7fb16ffff700 1 -- 172.21.15.32:0/1553077106 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c6ebf7b480 con 0x55c6ebfce680 2023-08-11T15:48:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.702+0000 7fb16ffff700 10 client.25504 handle_command_reply: tid=0 2023-08-11T15:48:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.702+0000 7fb1aa538b80 1 client.25504 shutdown 2023-08-11T15:48:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.702+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 msgr2=0x55c6ebfc4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.702+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfc4190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb184000c00 tx=0x7fb1840038b0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 shutdown_connections 2023-08-11T15:48:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ebfce680 0x55c6ebfd0aa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfac630 0x55c6ebfc4190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb16ffff700 0 client.25504 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1553077106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 >> 172.21.15.32:0/1553077106 conn(0x55c6ebed98f0 msgr2=0x55c6ebfa65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:03.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 shutdown_connections 2023-08-11T15:48:03.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1553077106 wait complete. 2023-08-11T15:48:03.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.703+0000 7fb1aa538b80 20 client.25504 trim_cache size 0 max 16384 2023-08-11T15:48:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.719+0000 7fb1aa538b80 1 Processor -- start 2023-08-11T15:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.719+0000 7fb1aa538b80 1 -- start start 2023-08-11T15:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfac630 0x55c6ebfaca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebfafab0 con 0x55c6ebfacff0 2023-08-11T15:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebfb0860 con 0x55c6ebfac630 2023-08-11T15:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ebfbe9b0 con 0x55c6ebfb1e80 2023-08-11T15:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19499b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfac630 0x55c6ebfaca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19519c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37156/0 (socket says 172.21.15.32:37156) 2023-08-11T15:48:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb18ffff700 1 -- 172.21.15.32:0/1109833621 learned_addr learned my addr 172.21.15.32:0/1109833621 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19499b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfac630 0x55c6ebfaca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:51656/0 (socket says 172.21.15.32:51656) 2023-08-11T15:48:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19519c700 1 -- 172.21.15.32:0/1109833621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfac630 msgr2=0x55c6ebfaca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19519c700 1 --2- 172.21.15.32:0/1109833621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfac630 0x55c6ebfaca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19519c700 1 -- 172.21.15.32:0/1109833621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 msgr2=0x55c6ebfc3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19519c700 1 --2- 172.21.15.32:0/1109833621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.720+0000 7fb19519c700 1 -- 172.21.15.32:0/1109833621 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ebd13c00 con 0x55c6ebfb1e80 2023-08-11T15:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.721+0000 7fb19519c700 1 --2- 172.21.15.32:0/1109833621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1880ca0e0 tx=0x7fb18800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a366364609543708 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.721+0000 7fb17b7fe700 1 -- 172.21.15.32:0/1109833621 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880999c0 con 0x55c6ebfb1e80 2023-08-11T15:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.721+0000 7fb17b7fe700 1 -- 172.21.15.32:0/1109833621 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb188004e60 con 0x55c6ebfb1e80 2023-08-11T15:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.721+0000 7fb17b7fe700 1 -- 172.21.15.32:0/1109833621 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18809a520 con 0x55c6ebfb1e80 2023-08-11T15:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1109833621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 msgr2=0x55c6ebfbe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1109833621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1880ca0e0 tx=0x7fb18800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1109833621 shutdown_connections 2023-08-11T15:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1109833621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ebfbe470 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1109833621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfac630 0x55c6ebfaca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/1109833621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfacff0 0x55c6ebfc3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1109833621 >> 172.21.15.32:0/1109833621 conn(0x55c6ebed98f0 msgr2=0x55c6ebfbeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1109833621 shutdown_connections 2023-08-11T15:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.722+0000 7fb1aa538b80 1 -- 172.21.15.32:0/1109833621 wait complete. 2023-08-11T15:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 -- start start 2023-08-11T15:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 Processor -- start 2023-08-11T15:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfac630 0x55c6ec016490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ec0169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ec021320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ec017110 con 0x55c6ebfac630 2023-08-11T15:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ec021860 con 0x55c6ebfacff0 2023-08-11T15:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ec0219f0 con 0x55c6ebfb1e80 2023-08-11T15:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb1aa538b80 10 client.0 _ll_register_callbacks cb 0x7fb17c013900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.723+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ec0169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb19499b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfac630 0x55c6ec016490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ec0169d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:51662/0 (socket says 172.21.15.32:51662) 2023-08-11T15:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 -- 172.21.15.32:0/2673517215 learned_addr learned my addr 172.21.15.32:0/2673517215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb19519c700 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ec021320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 -- 172.21.15.32:0/2673517215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 msgr2=0x55c6ec021320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ec021320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 -- 172.21.15.32:0/2673517215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfac630 msgr2=0x55c6ec016490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfac630 0x55c6ec016490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 -- 172.21.15.32:0/2673517215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1901a5ad0 con 0x55c6ebfacff0 2023-08-11T15:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb18ffff700 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ec0169d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005e10 tx=0x7fb17c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c80d69e6b12ed45 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.724+0000 7fb16ffff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.725+0000 7fb16ffff700 1 -- 172.21.15.32:0/2673517215 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c013350 con 0x55c6ebfacff0 2023-08-11T15:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.725+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb184004390 con 0x55c6ebfacff0 2023-08-11T15:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.725+0000 7fb16ffff700 1 -- 172.21.15.32:0/2673517215 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb17c00fdc0 con 0x55c6ebfacff0 2023-08-11T15:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.725+0000 7fb16ffff700 1 -- 172.21.15.32:0/2673517215 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb17c004410 con 0x55c6ebfacff0 2023-08-11T15:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.725+0000 7fb1aa538b80 10 client.25511 fetch_fsmap learned FSMap version 23 2023-08-11T15:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.725+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c6ebd511f0 con 0x55c6ebfacff0 2023-08-11T15:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.725+0000 7fb16ffff700 1 -- 172.21.15.32:0/2673517215 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb184004390 con 0x55c6ebfacff0 2023-08-11T15:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb16ffff700 1 -- 172.21.15.32:0/2673517215 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb17c003680 con 0x55c6ebfacff0 2023-08-11T15:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb1aa538b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 23 2023-08-11T15:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb1aa538b80 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb1aa538b80 20 client.25511 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ec0231b0 0x55c6ec0255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb1aa538b80 4 client.25511 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6ebef77c0 con 0x55c6ec0231b0 2023-08-11T15:48:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb19499b700 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ec0231b0 0x55c6ec0255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb19499b700 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ec0231b0 0x55c6ec0255d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a343468c9ea447a server_cookie=d68b5d845527425c in_seq=0 out_seq=0 2023-08-11T15:48:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:03.726+0000 7fb16ffff700 10 client.25511 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:03.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 50224 15.19 MB/sec warmup 19 sec latency 11.166 ms 2023-08-11T15:48:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb16ffff700 1 -- 172.21.15.32:0/2673517215 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6ebef77c0 con 0x55c6ec0231b0 2023-08-11T15:48:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb16ffff700 10 client.25511 handle_command_reply: tid=0 2023-08-11T15:48:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 client.25511 shutdown 2023-08-11T15:48:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 msgr2=0x55c6ec0169d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ec0169d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005e10 tx=0x7fb17c004750 comp rx=0 tx=0).stop 2023-08-11T15:48:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 shutdown_connections 2023-08-11T15:48:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55c6ec0231b0 0x55c6ec0255d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6ebfac630 0x55c6ec016490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6ebfb1e80 0x55c6ec021320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb16ffff700 0 client.25511 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 --2- 172.21.15.32:0/2673517215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6ebfacff0 0x55c6ec0169d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 >> 172.21.15.32:0/2673517215 conn(0x55c6ebed98f0 msgr2=0x55c6ebfb0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 shutdown_connections 2023-08-11T15:48:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.047+0000 7fb1aa538b80 1 -- 172.21.15.32:0/2673517215 wait complete. 2023-08-11T15:48:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb1aa538b80 20 client.25511 trim_cache size 0 max 16384 2023-08-11T15:48:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:48:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:48:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:48:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:48:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb1800a01e0 msgr2=0x7fb1800a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb1800a01e0 0x7fb1800a26a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55c6ebfa7e90 tx=0x7fb17c009cc0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901068c0 msgr2=0x7fb190198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901068c0 0x7fb190198940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb188009850 tx=0x7fb1880077f0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 shutdown_connections 2023-08-11T15:48:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb1800a01e0 0x7fb1800a26a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1901008b0 0x7fb19019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1901068c0 0x7fb190198940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 --2- 172.21.15.32:0/462018990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb190101270 0x7fb19019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 >> 172.21.15.32:0/462018990 conn(0x7fb1900fc380 msgr2=0x7fb1900fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.048+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 shutdown_connections 2023-08-11T15:48:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:04.049+0000 7fb196bff700 1 -- 172.21.15.32:0/462018990 wait complete. 2023-08-11T15:48:04.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:04 smithi032 ceph-mon[97103]: pgmap v45: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.0 MiB/s wr, 409 op/s 2023-08-11T15:48:04.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:04 smithi192 ceph-mon[130806]: pgmap v45: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.0 MiB/s wr, 409 op/s 2023-08-11T15:48:04.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:04 smithi134 ceph-mon[130447]: pgmap v45: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.0 MiB/s wr, 409 op/s 2023-08-11T15:48:04.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 52302 14.84 MB/sec warmup 20 sec latency 148.091 ms 2023-08-11T15:48:05.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 54845 14.90 MB/sec warmup 21 sec latency 7.852 ms 2023-08-11T15:48:06.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:06 smithi032 ceph-mon[97103]: pgmap v46: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2023-08-11T15:48:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:06 smithi192 ceph-mon[130806]: pgmap v46: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2023-08-11T15:48:06.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:06 smithi134 ceph-mon[130447]: pgmap v46: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2023-08-11T15:48:06.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 57593 14.88 MB/sec warmup 22 sec latency 22.617 ms 2023-08-11T15:48:07.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 60767 15.04 MB/sec warmup 23 sec latency 12.192 ms 2023-08-11T15:48:08.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:07 smithi134 ceph-mon[130447]: pgmap v47: 129 pgs: 129 active+clean; 17 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.9 MiB/s wr, 345 op/s 2023-08-11T15:48:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:07 smithi032 ceph-mon[97103]: pgmap v47: 129 pgs: 129 active+clean; 17 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.9 MiB/s wr, 345 op/s 2023-08-11T15:48:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:07 smithi192 ceph-mon[130806]: pgmap v47: 129 pgs: 129 active+clean; 17 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.9 MiB/s wr, 345 op/s 2023-08-11T15:48:08.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 63855 15.14 MB/sec warmup 24 sec latency 23.258 ms 2023-08-11T15:48:09.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 66466 14.93 MB/sec warmup 25 sec latency 6.175 ms 2023-08-11T15:48:10.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:10 smithi032 ceph-mon[97103]: pgmap v48: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 365 op/s 2023-08-11T15:48:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:10 smithi192 ceph-mon[130806]: pgmap v48: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 365 op/s 2023-08-11T15:48:10.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:10 smithi134 ceph-mon[130447]: pgmap v48: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 365 op/s 2023-08-11T15:48:10.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68131 14.79 MB/sec warmup 26 sec latency 127.853 ms 2023-08-11T15:48:11.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 69876 14.50 MB/sec warmup 27 sec latency 216.530 ms 2023-08-11T15:48:12.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:12 smithi032 ceph-mon[97103]: pgmap v49: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 359 op/s 2023-08-11T15:48:12.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:12 smithi192 ceph-mon[130806]: pgmap v49: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 359 op/s 2023-08-11T15:48:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:12 smithi134 ceph-mon[130447]: pgmap v49: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 359 op/s 2023-08-11T15:48:12.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 72142 14.48 MB/sec warmup 28 sec latency 8.413 ms 2023-08-11T15:48:13.733 INFO:tasks.workunit.client.0.smithi032.stdout: 1 75069 14.57 MB/sec warmup 29 sec latency 20.854 ms 2023-08-11T15:48:14.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:14 smithi032 ceph-mon[97103]: pgmap v50: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2023-08-11T15:48:14.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:14 smithi032 ceph-mon[97103]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-08-11T15:48:14.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:14 smithi192 ceph-mon[130806]: pgmap v50: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2023-08-11T15:48:14.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:14 smithi192 ceph-mon[130806]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-08-11T15:48:14.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:14 smithi134 ceph-mon[130447]: pgmap v50: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2023-08-11T15:48:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:14 smithi134 ceph-mon[130447]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-08-11T15:48:14.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 78195 14.68 MB/sec warmup 30 sec latency 17.314 ms 2023-08-11T15:48:15.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 80484 14.47 MB/sec warmup 31 sec latency 115.697 ms 2023-08-11T15:48:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:16 smithi032 ceph-mon[97103]: pgmap v51: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.1 MiB/s wr, 395 op/s 2023-08-11T15:48:16.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:16 smithi192 ceph-mon[130806]: pgmap v51: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.1 MiB/s wr, 395 op/s 2023-08-11T15:48:16.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:16 smithi134 ceph-mon[130447]: pgmap v51: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.1 MiB/s wr, 395 op/s 2023-08-11T15:48:16.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 81001 14.19 MB/sec warmup 32 sec latency 299.217 ms 2023-08-11T15:48:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 -- 172.21.15.32:0/4139244340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6438072cf0 msgr2=0x7f64380730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 --2- 172.21.15.32:0/4139244340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6438072cf0 0x7f64380730d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f64340098f0 tx=0x7f6434009c00 comp rx=0 tx=0).stop 2023-08-11T15:48:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 -- 172.21.15.32:0/4139244340 shutdown_connections 2023-08-11T15:48:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 --2- 172.21.15.32:0/4139244340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6438072cf0 0x7f64380730d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 --2- 172.21.15.32:0/4139244340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f643810d140 0x7f643810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 --2- 172.21.15.32:0/4139244340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438073610 0x7f643810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 -- 172.21.15.32:0/4139244340 >> 172.21.15.32:0/4139244340 conn(0x7f64380fc3d0 msgr2=0x7f64380fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 -- 172.21.15.32:0/4139244340 shutdown_connections 2023-08-11T15:48:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.207+0000 7f643f3ff700 1 -- 172.21.15.32:0/4139244340 wait complete. 2023-08-11T15:48:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.208+0000 7f643f3ff700 1 Processor -- start 2023-08-11T15:48:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.208+0000 7f643f3ff700 1 -- start start 2023-08-11T15:48:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643f3ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438072cf0 0x7f643819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643f3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438073610 0x7f643819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643f3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643810d140 0x7f6438198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643f3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438199130 con 0x7f643810d140 2023-08-11T15:48:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643f3ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64381992a0 con 0x7f6438072cf0 2023-08-11T15:48:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643f3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438199460 con 0x7f6438073610 2023-08-11T15:48:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438072cf0 0x7f643819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438072cf0 0x7f643819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:40008/0 (socket says 172.21.15.32:40008) 2023-08-11T15:48:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d19b700 1 -- 172.21.15.32:0/2463692833 learned_addr learned my addr 172.21.15.32:0/2463692833 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643c99a700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438073610 0x7f643819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d99c700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643810d140 0x7f6438198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d19b700 1 -- 172.21.15.32:0/2463692833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438073610 msgr2=0x7f643819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d19b700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438073610 0x7f643819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d19b700 1 -- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643810d140 msgr2=0x7f6438198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.209+0000 7f643d19b700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643810d140 0x7f6438198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.210+0000 7f643d19b700 1 -- 172.21.15.32:0/2463692833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434009520 con 0x7f6438072cf0 2023-08-11T15:48:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.210+0000 7f643d19b700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438072cf0 0x7f643819ec60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6434009810 tx=0x7f643400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.210+0000 7f642e7fc700 1 -- 172.21.15.32:0/2463692833 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643401c070 con 0x7f6438072cf0 2023-08-11T15:48:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.210+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64381996e0 con 0x7f6438072cf0 2023-08-11T15:48:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.210+0000 7f642e7fc700 1 -- 172.21.15.32:0/2463692833 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643400bb20 con 0x7f6438072cf0 2023-08-11T15:48:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.210+0000 7f642e7fc700 1 -- 172.21.15.32:0/2463692833 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434004930 con 0x7f6438072cf0 2023-08-11T15:48:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.210+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64381a5c50 con 0x7f6438072cf0 2023-08-11T15:48:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.214+0000 7f642e7fc700 1 -- 172.21.15.32:0/2463692833 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f6434020070 con 0x7f6438072cf0 2023-08-11T15:48:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.214+0000 7f6452dd2b80 1 Processor -- start 2023-08-11T15:48:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f6452dd2b80 1 -- start start 2023-08-11T15:48:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb10260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb0aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb16c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cda635d0 con 0x5596cdb0a630 2023-08-11T15:48:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cda60ef0 con 0x5596cdb0aff0 2023-08-11T15:48:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cda5c7e0 con 0x5596cdb0fe80 2023-08-11T15:48:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f643d99c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb10260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f643c99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb16c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f643d19b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb0aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f643d99c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb10260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39264/0 (socket says 172.21.15.32:39264) 2023-08-11T15:48:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.215+0000 7f643d99c700 1 -- 172.21.15.32:0/2340550371 learned_addr learned my addr 172.21.15.32:0/2340550371 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643d99c700 1 -- 172.21.15.32:0/2340550371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 msgr2=0x5596cdb16c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643d99c700 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb16c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f642e7fc700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64240a00c0 0x7f64240a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643d99c700 1 -- 172.21.15.32:0/2340550371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 msgr2=0x5596cdb0aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643d99c700 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb0aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643d99c700 1 -- 172.21.15.32:0/2340550371 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596cd8dc5d0 con 0x5596cdb0fe80 2023-08-11T15:48:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643c99a700 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb16c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f642e7fc700 1 -- 172.21.15.32:0/2463692833 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f64340cc5f0 con 0x7f6438072cf0 2023-08-11T15:48:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643d99c700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64240a00c0 0x7f64240a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f643d99c700 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb10260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64300115c0 tx=0x7f64300099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ce481dfc8bf088 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.216+0000 7f64237fe700 1 -- 172.21.15.32:0/2340550371 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000e040 con 0x5596cdb0fe80 2023-08-11T15:48:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f64237fe700 1 -- 172.21.15.32:0/2340550371 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643000f040 con 0x5596cdb0fe80 2023-08-11T15:48:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f64237fe700 1 -- 172.21.15.32:0/2340550371 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430025070 con 0x5596cdb0fe80 2023-08-11T15:48:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2340550371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 msgr2=0x5596cdb10260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb10260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64300115c0 tx=0x7f64300099e0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f643d99c700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64240a00c0 0x7f64240a2580 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6430020000 tx=0x7f6430023000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2340550371 shutdown_connections 2023-08-11T15:48:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb16c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb10260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2340550371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb0aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.217+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2340550371 >> 172.21.15.32:0/2340550371 conn(0x5596cda378f0 msgr2=0x5596cdb04e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.218+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2340550371 shutdown_connections 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.218+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2340550371 wait complete. 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.218+0000 7f6452dd2b80 1 -- start start 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.218+0000 7f6452dd2b80 1 Processor -- start 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb22270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0aff0 0x5596cdb227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0fe80 0x5596cdb1c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd9faa60 con 0x5596cdb0a630 2023-08-11T15:48:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd9fa760 con 0x5596cdb0fe80 2023-08-11T15:48:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd8de8c0 con 0x5596cdb0aff0 2023-08-11T15:48:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f6452dd2b80 10 client.0 _ll_register_callbacks cb 0x5596cd9fda50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f643d19b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb22270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f643c99a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0aff0 0x5596cdb227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f643d99c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0fe80 0x5596cdb1c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f643d19b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb22270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34676/0 (socket says 172.21.15.32:34676) 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.219+0000 7f643d19b700 1 -- 172.21.15.32:0/2115473955 learned_addr learned my addr 172.21.15.32:0/2115473955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f643d19b700 1 -- 172.21.15.32:0/2115473955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0aff0 msgr2=0x5596cdb227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f643d19b700 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0aff0 0x5596cdb227b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f643d19b700 1 -- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0fe80 msgr2=0x5596cdb1c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f643d19b700 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0fe80 0x5596cdb1c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f643d19b700 1 -- 172.21.15.32:0/2115473955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64280098e0 con 0x5596cdb0a630 2023-08-11T15:48:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f643d99c700 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0fe80 0x5596cdb1c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f643d19b700 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb22270 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6434009fd0 tx=0x7f643400bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33deb2dff414a58 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f6420ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:48:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f6420ff9700 1 -- 172.21.15.32:0/2115473955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434099850 con 0x5596cdb0a630 2023-08-11T15:48:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f6420ff9700 1 -- 172.21.15.32:0/2115473955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f643409a180 con 0x5596cdb0a630 2023-08-11T15:48:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596cdafd7e0 con 0x5596cdb0a630 2023-08-11T15:48:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f6420ff9700 1 -- 172.21.15.32:0/2115473955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64340995c0 con 0x5596cdb0a630 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f6452dd2b80 10 client.15516 fetch_fsmap learned FSMap version 23 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.220+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f6430011270 con 0x5596cdb0a630 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6420ff9700 1 -- 172.21.15.32:0/2115473955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596cdafd7e0 con 0x5596cdb0a630 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6420ff9700 1 -- 172.21.15.32:0/2115473955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64340d4070 con 0x5596cdb0a630 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6452dd2b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 23 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6452dd2b80 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6452dd2b80 20 client.15516 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb2c460 0x5596cdb2e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6452dd2b80 4 client.15516 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596cdad9480 con 0x5596cdb2c460 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.221+0000 7f643c99a700 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb2c460 0x5596cdb2e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.222+0000 7f643c99a700 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb2c460 0x5596cdb2e850 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9e8a7e86b58252 server_cookie=ce618ea61435922b in_seq=0 out_seq=0 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.222+0000 7f6420ff9700 10 client.15516 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.222+0000 7f6420ff9700 1 -- 172.21.15.32:0/2115473955 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5596cdad9480 con 0x5596cdb2c460 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.222+0000 7f6420ff9700 10 client.15516 handle_command_reply: tid=0 2023-08-11T15:48:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.222+0000 7f6452dd2b80 1 client.15516 shutdown 2023-08-11T15:48:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 msgr2=0x5596cdb22270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb22270 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6434009fd0 tx=0x7f643400bc90 comp rx=0 tx=0).stop 2023-08-11T15:48:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 shutdown_connections 2023-08-11T15:48:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb2c460 0x5596cdb2e850 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb22270 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0fe80 0x5596cdb1c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6420ff9700 0 client.15516 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/2115473955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0aff0 0x5596cdb227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 >> 172.21.15.32:0/2115473955 conn(0x5596cda378f0 msgr2=0x5596cdb045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 shutdown_connections 2023-08-11T15:48:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 1 -- 172.21.15.32:0/2115473955 wait complete. 2023-08-11T15:48:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.223+0000 7f6452dd2b80 20 client.15516 trim_cache size 0 max 16384 2023-08-11T15:48:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.239+0000 7f6452dd2b80 1 Processor -- start 2023-08-11T15:48:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.239+0000 7f6452dd2b80 1 -- start start 2023-08-11T15:48:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0a630 0x5596cdb1c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb21dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0fe80 0x5596cdb04d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cdb1d550 con 0x5596cdb0fe80 2023-08-11T15:48:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cdb0b3d0 con 0x5596cdb0aff0 2023-08-11T15:48:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cdafd570 con 0x5596cdb0a630 2023-08-11T15:48:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0a630 0x5596cdb1c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643c99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb21dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d99c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0fe80 0x5596cdb04d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0a630 0x5596cdb1c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:39284/0 (socket says 172.21.15.32:39284) 2023-08-11T15:48:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 -- 172.21.15.32:0/3976636313 learned_addr learned my addr 172.21.15.32:0/3976636313 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 -- 172.21.15.32:0/3976636313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 msgr2=0x5596cdb21dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 --2- 172.21.15.32:0/3976636313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb21dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 -- 172.21.15.32:0/3976636313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0fe80 msgr2=0x5596cdb04d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 --2- 172.21.15.32:0/3976636313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0fe80 0x5596cdb04d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.240+0000 7f643d19b700 1 -- 172.21.15.32:0/3976636313 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596cd888d80 con 0x5596cdb0a630 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f643d19b700 1 --2- 172.21.15.32:0/3976636313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0a630 0x5596cdb1c550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64340964f0 tx=0x7f643400fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae1c7261e2c3f61d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6423fff700 1 -- 172.21.15.32:0/3976636313 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340cc460 con 0x5596cdb0a630 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6423fff700 1 -- 172.21.15.32:0/3976636313 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64340995c0 con 0x5596cdb0a630 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6423fff700 1 -- 172.21.15.32:0/3976636313 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643409a180 con 0x5596cdb0a630 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3976636313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0a630 msgr2=0x5596cdb1c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3976636313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0a630 0x5596cdb1c550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64340964f0 tx=0x7f643400fa70 comp rx=0 tx=0).stop 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3976636313 shutdown_connections 2023-08-11T15:48:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3976636313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0fe80 0x5596cdb04d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3976636313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0a630 0x5596cdb1c550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3976636313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb21dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.241+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3976636313 >> 172.21.15.32:0/3976636313 conn(0x5596cda378f0 msgr2=0x5596cdb1be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.242+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3976636313 shutdown_connections 2023-08-11T15:48:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.242+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3976636313 wait complete. 2023-08-11T15:48:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.242+0000 7f6452dd2b80 1 -- start start 2023-08-11T15:48:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.242+0000 7f6452dd2b80 1 Processor -- start 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb74260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f6452dd2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb7f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cdb74eb0 con 0x5596cdb0a630 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cdb7f630 con 0x5596cdb0aff0 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f6452dd2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cdb7f7a0 con 0x5596cdb0fe80 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f6452dd2b80 10 client.0 _ll_register_callbacks cb 0x7f64340d2240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb747a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:40042/0 (socket says 172.21.15.32:40042) 2023-08-11T15:48:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 -- 172.21.15.32:0/3107728484 learned_addr learned my addr 172.21.15.32:0/3107728484 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643d99c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb7f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643d19b700 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb74260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 -- 172.21.15.32:0/3107728484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 msgr2=0x5596cdb7f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb7f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 -- 172.21.15.32:0/3107728484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 msgr2=0x5596cdb74260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb74260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643c99a700 1 -- 172.21.15.32:0/3107728484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6430011270 con 0x5596cdb0aff0 2023-08-11T15:48:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.243+0000 7f643d99c700 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb7f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f643c99a700 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb747a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5596cdb05a30 tx=0x7f64280043d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c449e3850ccc2d47 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f6420ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:48:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f6420ff9700 1 -- 172.21.15.32:0/3107728484 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428016070 con 0x5596cdb0aff0 2023-08-11T15:48:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f643000f040 con 0x5596cdb0aff0 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f6420ff9700 1 -- 172.21.15.32:0/3107728484 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f642801d8c0 con 0x5596cdb0aff0 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f6420ff9700 1 -- 172.21.15.32:0/3107728484 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f642801da80 con 0x5596cdb0aff0 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f6452dd2b80 10 client.25529 fetch_fsmap learned FSMap version 23 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.244+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5596cd871c00 con 0x5596cdb0aff0 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6420ff9700 1 -- 172.21.15.32:0/3107728484 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643000f040 con 0x5596cdb0aff0 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6420ff9700 1 -- 172.21.15.32:0/3107728484 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6428025410 con 0x5596cdb0aff0 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6452dd2b80 10 client.25529 fetch_fsmap finished waiting for FSMap version 23 2023-08-11T15:48:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6452dd2b80 10 client.25529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6452dd2b80 20 client.25529 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb80fb0 0x5596cdb833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6452dd2b80 4 client.25529 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:48:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596cda557c0 con 0x5596cdb80fb0 2023-08-11T15:48:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.245+0000 7f643d19b700 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb80fb0 0x5596cdb833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.246+0000 7f6420ff9700 10 client.25529 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.246+0000 7f643d19b700 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb80fb0 0x5596cdb833d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc27c79ebea8df27 server_cookie=f4bb24180194841b in_seq=0 out_seq=0 2023-08-11T15:48:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.375+0000 7f6420ff9700 1 -- 172.21.15.32:0/3107728484 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596cda557c0 con 0x5596cdb80fb0 2023-08-11T15:48:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.375+0000 7f6420ff9700 10 client.25529 handle_command_reply: tid=0 2023-08-11T15:48:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.375+0000 7f6452dd2b80 1 client.25529 shutdown 2023-08-11T15:48:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 msgr2=0x5596cdb747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb747a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5596cdb05a30 tx=0x7f64280043d0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 shutdown_connections 2023-08-11T15:48:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5596cdb80fb0 0x5596cdb833d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596cdb0a630 0x5596cdb74260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596cdb0fe80 0x5596cdb7f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 --2- 172.21.15.32:0/3107728484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596cdb0aff0 0x5596cdb747a0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5596cdb05a30 tx=0x7f64280043d0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 >> 172.21.15.32:0/3107728484 conn(0x5596cda378f0 msgr2=0x5596cdb1be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.376+0000 7f6420ff9700 0 client.25529 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.377+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 shutdown_connections 2023-08-11T15:48:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.377+0000 7f6452dd2b80 1 -- 172.21.15.32:0/3107728484 wait complete. 2023-08-11T15:48:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.378+0000 7f6452dd2b80 20 client.25529 trim_cache size 0 max 16384 2023-08-11T15:48:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:48:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:48:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:48:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:48:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64240a00c0 msgr2=0x7f64240a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64240a00c0 0x7f64240a2580 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6430020000 tx=0x7f6430023000 comp rx=0 tx=0).stop 2023-08-11T15:48:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438072cf0 msgr2=0x7f643819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438072cf0 0x7f643819ec60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6434009810 tx=0x7f643400fa00 comp rx=0 tx=0).stop 2023-08-11T15:48:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 shutdown_connections 2023-08-11T15:48:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64240a00c0 0x7f64240a2580 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643810d140 0x7f6438198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6438072cf0 0x7f643819ec60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 --2- 172.21.15.32:0/2463692833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6438073610 0x7f643819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 >> 172.21.15.32:0/2463692833 conn(0x7f64380fc3d0 msgr2=0x7f643810f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 shutdown_connections 2023-08-11T15:48:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:17.379+0000 7f643f3ff700 1 -- 172.21.15.32:0/2463692833 wait complete. 2023-08-11T15:48:17.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 82708 14.02 MB/sec warmup 33 sec latency 59.587 ms 2023-08-11T15:48:17.803 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:48:17 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setuser ceph since I am not root 2023-08-11T15:48:17.803 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:48:17 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setgroup ceph since I am not root 2023-08-11T15:48:17.803 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:48:17 smithi032 ceph-mds[131163]: -- 172.21.15.32:0/422581116 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562f62ab1e0 con 0x5562f6280c00 2023-08-11T15:48:17.804 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:48:17 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: starting mds.a at 2023-08-11T15:48:18.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:17 smithi134 ceph-mon[130447]: pgmap v52: 129 pgs: 129 active+clean; 42 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.9 MiB/s wr, 538 op/s 2023-08-11T15:48:18.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:17 smithi134 ceph-mon[130447]: daemon mds.a restarted 2023-08-11T15:48:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:17 smithi032 ceph-mon[97103]: pgmap v52: 129 pgs: 129 active+clean; 42 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.9 MiB/s wr, 538 op/s 2023-08-11T15:48:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:17 smithi032 ceph-mon[97103]: daemon mds.a restarted 2023-08-11T15:48:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:17 smithi192 ceph-mon[130806]: pgmap v52: 129 pgs: 129 active+clean; 42 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.9 MiB/s wr, 538 op/s 2023-08-11T15:48:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:17 smithi192 ceph-mon[130806]: daemon mds.a restarted 2023-08-11T15:48:18.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85785 14.12 MB/sec warmup 34 sec latency 9.964 ms 2023-08-11T15:48:19.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:18 smithi134 ceph-mon[130447]: osdmap e95: 12 total, 12 up, 12 in 2023-08-11T15:48:19.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:18 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6840/633787060,v1:172.21.15.32:6841/633787060] up:boot 2023-08-11T15:48:19.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:18 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:48:19.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:18 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:48:19.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:18 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:48:19.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:18 smithi032 ceph-mon[97103]: osdmap e95: 12 total, 12 up, 12 in 2023-08-11T15:48:19.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:18 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6840/633787060,v1:172.21.15.32:6841/633787060] up:boot 2023-08-11T15:48:19.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:18 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:48:19.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:18 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:48:19.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:18 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:48:19.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:18 smithi192 ceph-mon[130806]: osdmap e95: 12 total, 12 up, 12 in 2023-08-11T15:48:19.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:18 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6840/633787060,v1:172.21.15.32:6841/633787060] up:boot 2023-08-11T15:48:19.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:18 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:48:19.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:18 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:48:19.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:18 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:48:19.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 88939 14.25 MB/sec warmup 35 sec latency 11.199 ms 2023-08-11T15:48:20.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:19 smithi134 ceph-mon[130447]: pgmap v54: 129 pgs: 129 active+clean; 42 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 440 op/s 2023-08-11T15:48:20.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:19 smithi032 ceph-mon[97103]: pgmap v54: 129 pgs: 129 active+clean; 42 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 440 op/s 2023-08-11T15:48:20.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:19 smithi192 ceph-mon[130806]: pgmap v54: 129 pgs: 129 active+clean; 42 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 440 op/s 2023-08-11T15:48:20.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 91619 14.19 MB/sec warmup 36 sec latency 15.466 ms 2023-08-11T15:48:21.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 93424 14.06 MB/sec warmup 37 sec latency 66.368 ms 2023-08-11T15:48:22.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:22 smithi032 ceph-mon[97103]: pgmap v55: 129 pgs: 129 active+clean; 42 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 440 op/s 2023-08-11T15:48:22.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:22 smithi192 ceph-mon[130806]: pgmap v55: 129 pgs: 129 active+clean; 42 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 440 op/s 2023-08-11T15:48:22.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:22 smithi134 ceph-mon[130447]: pgmap v55: 129 pgs: 129 active+clean; 42 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 440 op/s 2023-08-11T15:48:22.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 96357 14.14 MB/sec warmup 38 sec latency 13.969 ms 2023-08-11T15:48:23.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 99565 14.26 MB/sec warmup 39 sec latency 6.253 ms 2023-08-11T15:48:24.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:24 smithi032 ceph-mon[97103]: pgmap v56: 129 pgs: 129 active+clean; 32 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 477 op/s 2023-08-11T15:48:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:24 smithi192 ceph-mon[130806]: pgmap v56: 129 pgs: 129 active+clean; 32 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 477 op/s 2023-08-11T15:48:24.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:24 smithi134 ceph-mon[130447]: pgmap v56: 129 pgs: 129 active+clean; 32 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 477 op/s 2023-08-11T15:48:24.735 INFO:tasks.workunit.client.0.smithi032.stdout: 1 101919 14.11 MB/sec warmup 40 sec latency 25.960 ms 2023-08-11T15:48:25.735 INFO:tasks.workunit.client.0.smithi032.stdout: 1 104313 14.16 MB/sec warmup 41 sec latency 21.735 ms 2023-08-11T15:48:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:26 smithi032 ceph-mon[97103]: pgmap v57: 129 pgs: 129 active+clean; 32 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 477 op/s 2023-08-11T15:48:26.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:26 smithi192 ceph-mon[130806]: pgmap v57: 129 pgs: 129 active+clean; 32 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 477 op/s 2023-08-11T15:48:26.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:26 smithi134 ceph-mon[130447]: pgmap v57: 129 pgs: 129 active+clean; 32 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 477 op/s 2023-08-11T15:48:26.735 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107139 14.16 MB/sec warmup 42 sec latency 10.120 ms 2023-08-11T15:48:27.735 INFO:tasks.workunit.client.0.smithi032.stdout: 1 110292 14.27 MB/sec warmup 43 sec latency 11.561 ms 2023-08-11T15:48:28.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:27 smithi134 ceph-mon[130447]: pgmap v58: 129 pgs: 129 active+clean; 35 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 337 op/s 2023-08-11T15:48:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:27 smithi032 ceph-mon[97103]: pgmap v58: 129 pgs: 129 active+clean; 35 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 337 op/s 2023-08-11T15:48:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:27 smithi192 ceph-mon[130806]: pgmap v58: 129 pgs: 129 active+clean; 35 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 337 op/s 2023-08-11T15:48:28.735 INFO:tasks.workunit.client.0.smithi032.stdout: 1 112969 14.24 MB/sec warmup 44 sec latency 28.220 ms 2023-08-11T15:48:29.735 INFO:tasks.workunit.client.0.smithi032.stdout: 1 115211 14.18 MB/sec warmup 45 sec latency 20.563 ms 2023-08-11T15:48:30.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:30 smithi032 ceph-mon[97103]: pgmap v59: 129 pgs: 129 active+clean; 39 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2023-08-11T15:48:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 -- 172.21.15.32:0/1483004776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec41068c0 msgr2=0x7fdec4106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 --2- 172.21.15.32:0/1483004776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec41068c0 0x7fdec4106ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009990 tx=0x7fdeb8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 -- 172.21.15.32:0/1483004776 shutdown_connections 2023-08-11T15:48:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 --2- 172.21.15.32:0/1483004776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec4101270 0x7fdec410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 --2- 172.21.15.32:0/1483004776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec41068c0 0x7fdec4106ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 --2- 172.21.15.32:0/1483004776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec41008b0 0x7fdec4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 -- 172.21.15.32:0/1483004776 >> 172.21.15.32:0/1483004776 conn(0x7fdec40fc380 msgr2=0x7fdec40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 -- 172.21.15.32:0/1483004776 shutdown_connections 2023-08-11T15:48:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.563+0000 7fdecabff700 1 -- 172.21.15.32:0/1483004776 wait complete. 2023-08-11T15:48:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.564+0000 7fdecabff700 1 Processor -- start 2023-08-11T15:48:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdecabff700 1 -- start start 2023-08-11T15:48:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdecabff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec41008b0 0x7fdec4072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdecabff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec4101270 0x7fdec4073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdecabff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec41068c0 0x7fdec41a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdecabff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41a50f0 con 0x7fdec41068c0 2023-08-11T15:48:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdecabff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41a5230 con 0x7fdec41008b0 2023-08-11T15:48:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdecabff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41a53a0 con 0x7fdec4101270 2023-08-11T15:48:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdec899b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec41008b0 0x7fdec4072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec4101270 0x7fdec4073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec4101270 0x7fdec4073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43248/0 (socket says 172.21.15.32:43248) 2023-08-11T15:48:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdec3fff700 1 -- 172.21.15.32:0/2833469637 learned_addr learned my addr 172.21.15.32:0/2833469637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdec899b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec41008b0 0x7fdec4072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44484/0 (socket says 172.21.15.32:44484) 2023-08-11T15:48:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.565+0000 7fdec919c700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec41068c0 0x7fdec41a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec3fff700 1 -- 172.21.15.32:0/2833469637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec41008b0 msgr2=0x7fdec4072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec3fff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec41008b0 0x7fdec4072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec3fff700 1 -- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec41068c0 msgr2=0x7fdec41a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec3fff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec41068c0 0x7fdec41a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec3fff700 1 -- 172.21.15.32:0/2833469637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80095f0 con 0x7fdec4101270 2023-08-11T15:48:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec919c700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec41068c0 0x7fdec41a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:48:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec3fff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec4101270 0x7fdec4073340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e9a0 tx=0x7fdeb000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec1ffb700 1 -- 172.21.15.32:0/2833469637 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0009790 con 0x7fdec4101270 2023-08-11T15:48:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec1ffb700 1 -- 172.21.15.32:0/2833469637 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeb000cbd0 con 0x7fdec4101270 2023-08-11T15:48:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec41a5680 con 0x7fdec4101270 2023-08-11T15:48:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdec1ffb700 1 -- 172.21.15.32:0/2833469637 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0018680 con 0x7fdec4101270 2023-08-11T15:48:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.566+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec41a5be0 con 0x7fdec4101270 2023-08-11T15:48:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.570+0000 7fdec1ffb700 1 -- 172.21.15.32:0/2833469637 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fdeb000cd40 con 0x7fdec4101270 2023-08-11T15:48:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 Processor -- start 2023-08-11T15:48:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 -- start start 2023-08-11T15:48:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdec1ffb700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fdeb40a0140 0x7fdeb40a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba671260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba66bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba677c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba5c45d0 con 0x55bdba66b630 2023-08-11T15:48:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba5c1ef0 con 0x55bdba670e80 2023-08-11T15:48:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdede48cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba5bd7e0 con 0x55bdba66bff0 2023-08-11T15:48:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.571+0000 7fdec1ffb700 1 -- 172.21.15.32:0/2833469637 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fdeb0014070 con 0x7fdec4101270 2023-08-11T15:48:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba671260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba671260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44496/0 (socket says 172.21.15.32:44496) 2023-08-11T15:48:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 -- 172.21.15.32:0/103997805 learned_addr learned my addr 172.21.15.32:0/103997805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec3fff700 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba66bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec919c700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fdeb40a0140 0x7fdeb40a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 -- 172.21.15.32:0/103997805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 msgr2=0x55bdba677c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec919c700 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba677c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba677c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 -- 172.21.15.32:0/103997805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 msgr2=0x55bdba66bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba66bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 -- 172.21.15.32:0/103997805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdba43d5d0 con 0x55bdba670e80 2023-08-11T15:48:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec919c700 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba677c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:48:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec3fff700 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba66bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.572+0000 7fdec899b700 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba671260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005ea0 tx=0x7fdeb8005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90fe229b752df3b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.573+0000 7fdeaeffd700 1 -- 172.21.15.32:0/103997805 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb801d070 con 0x55bdba670e80 2023-08-11T15:48:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.573+0000 7fdec919c700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fdeb40a0140 0x7fdeb40a2600 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55bdba666a30 tx=0x7fdebc009920 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.573+0000 7fdeaeffd700 1 -- 172.21.15.32:0/103997805 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeb801e040 con 0x55bdba670e80 2023-08-11T15:48:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.573+0000 7fdeaeffd700 1 -- 172.21.15.32:0/103997805 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80048f0 con 0x55bdba670e80 2023-08-11T15:48:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.573+0000 7fdede48cb80 1 -- 172.21.15.32:0/103997805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 msgr2=0x55bdba671260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.573+0000 7fdede48cb80 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba671260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005ea0 tx=0x7fdeb8005ed0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 -- 172.21.15.32:0/103997805 shutdown_connections 2023-08-11T15:48:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba677c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba671260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 --2- 172.21.15.32:0/103997805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba66bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 -- 172.21.15.32:0/103997805 >> 172.21.15.32:0/103997805 conn(0x55bdba5988f0 msgr2=0x55bdba665e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 -- 172.21.15.32:0/103997805 shutdown_connections 2023-08-11T15:48:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 -- 172.21.15.32:0/103997805 wait complete. 2023-08-11T15:48:30.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:30 smithi192 ceph-mon[130806]: pgmap v59: 129 pgs: 129 active+clean; 39 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2023-08-11T15:48:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.574+0000 7fdede48cb80 1 -- start start 2023-08-11T15:48:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 1 Processor -- start 2023-08-11T15:48:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba67a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba67aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba674ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba55ba60 con 0x55bdba66b630 2023-08-11T15:48:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba55b760 con 0x55bdba670e80 2023-08-11T15:48:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba43f8c0 con 0x55bdba66bff0 2023-08-11T15:48:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdede48cb80 10 client.0 _ll_register_callbacks cb 0x55bdba55ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdec899b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba67a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdec919c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba674ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdec899b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba67a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50566/0 (socket says 172.21.15.32:50566) 2023-08-11T15:48:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.575+0000 7fdec899b700 1 -- 172.21.15.32:0/1735248736 learned_addr learned my addr 172.21.15.32:0/1735248736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec3fff700 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba67aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec899b700 1 -- 172.21.15.32:0/1735248736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 msgr2=0x55bdba67aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec899b700 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba67aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec899b700 1 -- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 msgr2=0x55bdba674ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec899b700 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba674ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec899b700 1 -- 172.21.15.32:0/1735248736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec41a5be0 con 0x55bdba66b630 2023-08-11T15:48:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec919c700 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba674ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdec899b700 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba67a970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009990 tx=0x7fdeb800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335cf47de9fbde68 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdea3fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:48:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdea3fff700 1 -- 172.21.15.32:0/1735248736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8031070 con 0x55bdba66b630 2023-08-11T15:48:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdea3fff700 1 -- 172.21.15.32:0/1735248736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fdeb8005980 con 0x55bdba66b630 2023-08-11T15:48:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.576+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdba65e7e0 con 0x55bdba66b630 2023-08-11T15:48:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdea3fff700 1 -- 172.21.15.32:0/1735248736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeb801d040 con 0x55bdba66b630 2023-08-11T15:48:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 10 client.15537 fetch_fsmap learned FSMap version 25 2023-08-11T15:48:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fdec410e080 con 0x55bdba66b630 2023-08-11T15:48:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdea3fff700 1 -- 172.21.15.32:0/1735248736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdba65e7e0 con 0x55bdba66b630 2023-08-11T15:48:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdea3fff700 1 -- 172.21.15.32:0/1735248736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdeb8034020 con 0x55bdba66b630 2023-08-11T15:48:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:48:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 20 client.15537 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6848a0 0x55bdba686c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 4 client.15537 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:48:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.577+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdba63a480 con 0x55bdba6848a0 2023-08-11T15:48:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.578+0000 7fdec3fff700 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6848a0 0x55bdba686c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.578+0000 7fdec3fff700 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6848a0 0x55bdba686c90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e6483a958a8f7 server_cookie=4fe16728eef32563 in_seq=0 out_seq=0 2023-08-11T15:48:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.578+0000 7fdea3fff700 10 client.15537 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.579+0000 7fdea3fff700 1 -- 172.21.15.32:0/1735248736 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55bdba63a480 con 0x55bdba6848a0 2023-08-11T15:48:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.579+0000 7fdea3fff700 10 client.15537 handle_command_reply: tid=0 2023-08-11T15:48:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.579+0000 7fdede48cb80 1 client.15537 shutdown 2023-08-11T15:48:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 msgr2=0x55bdba67a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba67a970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009990 tx=0x7fdeb800fc00 comp rx=0 tx=0).stop 2023-08-11T15:48:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 shutdown_connections 2023-08-11T15:48:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6848a0 0x55bdba686c90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66b630 0x55bdba67a970 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba674ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1735248736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66bff0 0x55bdba67aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdea3fff700 0 client.15537 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 >> 172.21.15.32:0/1735248736 conn(0x55bdba5988f0 msgr2=0x55bdba6655d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 shutdown_connections 2023-08-11T15:48:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 1 -- 172.21.15.32:0/1735248736 wait complete. 2023-08-11T15:48:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.580+0000 7fdede48cb80 20 client.15537 trim_cache size 0 max 16384 2023-08-11T15:48:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.595+0000 7fdede48cb80 1 Processor -- start 2023-08-11T15:48:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.595+0000 7fdede48cb80 1 -- start start 2023-08-11T15:48:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba66b630 0x55bdba67a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba665d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba670e80 0x55bdba66fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdede48cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba666a60 con 0x55bdba66bff0 2023-08-11T15:48:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdede48cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba66f900 con 0x55bdba66b630 2023-08-11T15:48:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdede48cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba66e860 con 0x55bdba670e80 2023-08-11T15:48:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec899b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba66b630 0x55bdba67a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec919c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba670e80 0x55bdba66fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba665d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec919c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba670e80 0x55bdba66fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43274/0 (socket says 172.21.15.32:43274) 2023-08-11T15:48:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec919c700 1 -- 172.21.15.32:0/2023242290 learned_addr learned my addr 172.21.15.32:0/2023242290 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec899b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba66b630 0x55bdba67a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44512/0 (socket says 172.21.15.32:44512) 2023-08-11T15:48:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec899b700 1 -- 172.21.15.32:0/2023242290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba670e80 msgr2=0x55bdba66fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.596+0000 7fdec899b700 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba670e80 0x55bdba66fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdec899b700 1 -- 172.21.15.32:0/2023242290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 msgr2=0x55bdba665d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdec899b700 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba665d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdec899b700 1 -- 172.21.15.32:0/2023242290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdba3e9d80 con 0x55bdba66b630 2023-08-11T15:48:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdec919c700 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba670e80 0x55bdba66fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdec3fff700 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba665d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdec899b700 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba66b630 0x55bdba67a4d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004860 tx=0x7fdeb8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30c3ed285b3abca server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdeaf7fe700 1 -- 172.21.15.32:0/2023242290 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8031070 con 0x55bdba66b630 2023-08-11T15:48:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdeaf7fe700 1 -- 172.21.15.32:0/2023242290 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeb8005bc0 con 0x55bdba66b630 2023-08-11T15:48:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.597+0000 7fdeaf7fe700 1 -- 172.21.15.32:0/2023242290 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80055a0 con 0x55bdba66b630 2023-08-11T15:48:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 -- 172.21.15.32:0/2023242290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba66b630 msgr2=0x55bdba67a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba66b630 0x55bdba67a4d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004860 tx=0x7fdeb8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 -- 172.21.15.32:0/2023242290 shutdown_connections 2023-08-11T15:48:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba670e80 0x55bdba66fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba66b630 0x55bdba67a4d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 --2- 172.21.15.32:0/2023242290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba665d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 -- 172.21.15.32:0/2023242290 >> 172.21.15.32:0/2023242290 conn(0x55bdba5988f0 msgr2=0x55bdba66ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 -- 172.21.15.32:0/2023242290 shutdown_connections 2023-08-11T15:48:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 -- 172.21.15.32:0/2023242290 wait complete. 2023-08-11T15:48:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.598+0000 7fdede48cb80 1 -- start start 2023-08-11T15:48:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 1 Processor -- start 2023-08-11T15:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66b630 0x55bdba6d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba6d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba679930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba679f10 con 0x55bdba66bff0 2023-08-11T15:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba67a080 con 0x55bdba670e80 2023-08-11T15:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba67a240 con 0x55bdba66b630 2023-08-11T15:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdede48cb80 10 client.0 _ll_register_callbacks cb 0x7fdeb0010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdec899b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66b630 0x55bdba6d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdec919c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba679930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdec899b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66b630 0x55bdba6d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43290/0 (socket says 172.21.15.32:43290) 2023-08-11T15:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdec899b700 1 -- 172.21.15.32:0/1938868951 learned_addr learned my addr 172.21.15.32:0/1938868951 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.599+0000 7fdec3fff700 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba6d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdec899b700 1 -- 172.21.15.32:0/1938868951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 msgr2=0x55bdba679930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdec899b700 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba679930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdec899b700 1 -- 172.21.15.32:0/1938868951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 msgr2=0x55bdba6d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdec899b700 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba6d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdec899b700 1 -- 172.21.15.32:0/1938868951 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec41a5680 con 0x55bdba66b630 2023-08-11T15:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdec899b700 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66b630 0x55bdba6d0fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800f7b0 tx=0x7fdeb800b300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0931365ed73e1dc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdea3fff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdea3fff700 1 -- 172.21.15.32:0/1938868951 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb801d070 con 0x55bdba66b630 2023-08-11T15:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.600+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdba5b67c0 con 0x55bdba66b630 2023-08-11T15:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdea3fff700 1 -- 172.21.15.32:0/1938868951 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fdeb8005990 con 0x55bdba66b630 2023-08-11T15:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdea3fff700 1 -- 172.21.15.32:0/1938868951 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeb8031040 con 0x55bdba66b630 2023-08-11T15:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 10 client.25549 fetch_fsmap learned FSMap version 25 2023-08-11T15:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fdebc013070 con 0x55bdba66b630 2023-08-11T15:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdea3fff700 1 -- 172.21.15.32:0/1938868951 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdba5b67c0 con 0x55bdba66b630 2023-08-11T15:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdea3fff700 1 -- 172.21.15.32:0/1938868951 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdeb80099c0 con 0x55bdba66b630 2023-08-11T15:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 10 client.25549 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 10 client.25549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 20 client.25549 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6d8a70 0x55bdba6dae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 4 client.25549 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.601+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdba576ff0 con 0x55bdba6d8a70 2023-08-11T15:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.602+0000 7fdec3fff700 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6d8a70 0x55bdba6dae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.602+0000 7fdec3fff700 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6d8a70 0x55bdba6dae60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1add03aadf06b7 server_cookie=3701ef0f63e4c4b4 in_seq=0 out_seq=0 2023-08-11T15:48:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.602+0000 7fdea3fff700 10 client.25549 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:30 smithi134 ceph-mon[130447]: pgmap v59: 129 pgs: 129 active+clean; 39 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2023-08-11T15:48:30.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 117545 14.17 MB/sec warmup 46 sec latency 71.985 ms 2023-08-11T15:48:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdea3fff700 1 -- 172.21.15.32:0/1938868951 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdba576ff0 con 0x55bdba6d8a70 2023-08-11T15:48:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdea3fff700 10 client.25549 handle_command_reply: tid=0 2023-08-11T15:48:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 client.25549 shutdown 2023-08-11T15:48:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66b630 msgr2=0x55bdba6d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66b630 0x55bdba6d0fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800f7b0 tx=0x7fdeb800b300 comp rx=0 tx=0).stop 2023-08-11T15:48:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 shutdown_connections 2023-08-11T15:48:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55bdba6d8a70 0x55bdba6dae60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba670e80 0x55bdba679930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bdba66b630 0x55bdba6d0fe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdea3fff700 0 client.25549 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 --2- 172.21.15.32:0/1938868951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bdba66bff0 0x55bdba6d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 >> 172.21.15.32:0/1938868951 conn(0x55bdba5988f0 msgr2=0x55bdba66fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 shutdown_connections 2023-08-11T15:48:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 1 -- 172.21.15.32:0/1938868951 wait complete. 2023-08-11T15:48:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.848+0000 7fdede48cb80 20 client.25549 trim_cache size 0 max 16384 2023-08-11T15:48:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:48:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:48:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:48:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:48:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fdeb40a0140 msgr2=0x7fdeb40a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fdeb40a0140 0x7fdeb40a2600 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55bdba666a30 tx=0x7fdebc009920 comp rx=0 tx=0).stop 2023-08-11T15:48:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec4101270 msgr2=0x7fdec4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec4101270 0x7fdec4073340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e9a0 tx=0x7fdeb000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 shutdown_connections 2023-08-11T15:48:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fdeb40a0140 0x7fdeb40a2600 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec41068c0 0x7fdec41a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdec41008b0 0x7fdec4072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 --2- 172.21.15.32:0/2833469637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdec4101270 0x7fdec4073340 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 >> 172.21.15.32:0/2833469637 conn(0x7fdec40fc380 msgr2=0x7fdec41043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 shutdown_connections 2023-08-11T15:48:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:30.849+0000 7fdecabff700 1 -- 172.21.15.32:0/2833469637 wait complete. 2023-08-11T15:48:31.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 120363 14.18 MB/sec warmup 47 sec latency 154.239 ms 2023-08-11T15:48:32.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:32 smithi032 ceph-mon[97103]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 260 op/s 2023-08-11T15:48:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:32 smithi192 ceph-mon[130806]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 260 op/s 2023-08-11T15:48:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:32 smithi134 ceph-mon[130447]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 260 op/s 2023-08-11T15:48:32.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 122716 14.10 MB/sec warmup 48 sec latency 33.633 ms 2023-08-11T15:48:33.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124995 14.10 MB/sec warmup 49 sec latency 14.874 ms 2023-08-11T15:48:34.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:34 smithi032 ceph-mon[97103]: pgmap v61: 129 pgs: 129 active+clean; 37 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 493 op/s 2023-08-11T15:48:34.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:34 smithi192 ceph-mon[130806]: pgmap v61: 129 pgs: 129 active+clean; 37 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 493 op/s 2023-08-11T15:48:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:34 smithi134 ceph-mon[130447]: pgmap v61: 129 pgs: 129 active+clean; 37 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 493 op/s 2023-08-11T15:48:34.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 127939 14.18 MB/sec warmup 50 sec latency 129.035 ms 2023-08-11T15:48:35.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 130486 14.13 MB/sec warmup 51 sec latency 23.107 ms 2023-08-11T15:48:36.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:36 smithi032 ceph-mon[97103]: pgmap v62: 129 pgs: 129 active+clean; 37 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-08-11T15:48:36.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:36 smithi192 ceph-mon[130806]: pgmap v62: 129 pgs: 129 active+clean; 37 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-08-11T15:48:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:36 smithi134 ceph-mon[130447]: pgmap v62: 129 pgs: 129 active+clean; 37 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-08-11T15:48:36.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 132808 14.11 MB/sec warmup 52 sec latency 20.591 ms 2023-08-11T15:48:37.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135285 14.11 MB/sec warmup 53 sec latency 88.499 ms 2023-08-11T15:48:38.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:37 smithi134 ceph-mon[130447]: pgmap v63: 129 pgs: 129 active+clean; 40 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 347 op/s 2023-08-11T15:48:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:37 smithi032 ceph-mon[97103]: pgmap v63: 129 pgs: 129 active+clean; 40 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 347 op/s 2023-08-11T15:48:38.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:37 smithi192 ceph-mon[130806]: pgmap v63: 129 pgs: 129 active+clean; 40 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 347 op/s 2023-08-11T15:48:38.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 137379 13.99 MB/sec warmup 54 sec latency 264.963 ms 2023-08-11T15:48:39.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 139311 13.96 MB/sec warmup 55 sec latency 48.009 ms 2023-08-11T15:48:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:40 smithi032 ceph-mon[97103]: pgmap v64: 129 pgs: 129 active+clean; 44 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 386 op/s 2023-08-11T15:48:40.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:40 smithi192 ceph-mon[130806]: pgmap v64: 129 pgs: 129 active+clean; 44 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 386 op/s 2023-08-11T15:48:40.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:40 smithi134 ceph-mon[130447]: pgmap v64: 129 pgs: 129 active+clean; 44 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 386 op/s 2023-08-11T15:48:40.737 INFO:tasks.workunit.client.0.smithi032.stdout: 1 142298 14.03 MB/sec warmup 56 sec latency 30.590 ms 2023-08-11T15:48:41.736 INFO:tasks.workunit.client.0.smithi032.stdout: 1 145271 14.04 MB/sec warmup 57 sec latency 21.388 ms 2023-08-11T15:48:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:42 smithi032 ceph-mon[97103]: pgmap v65: 129 pgs: 129 active+clean; 44 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 327 op/s 2023-08-11T15:48:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:42 smithi192 ceph-mon[130806]: pgmap v65: 129 pgs: 129 active+clean; 44 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 327 op/s 2023-08-11T15:48:42.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:42 smithi134 ceph-mon[130447]: pgmap v65: 129 pgs: 129 active+clean; 44 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 327 op/s 2023-08-11T15:48:42.737 INFO:tasks.workunit.client.0.smithi032.stdout: 1 147757 14.00 MB/sec warmup 58 sec latency 10.544 ms 2023-08-11T15:48:43.737 INFO:tasks.workunit.client.0.smithi032.stdout: 1 149755 13.97 MB/sec warmup 59 sec latency 80.562 ms 2023-08-11T15:48:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 -- 172.21.15.32:0/2898246017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa060101270 msgr2=0x7fa06010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 --2- 172.21.15.32:0/2898246017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa060101270 0x7fa06010d780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa058009f90 tx=0x7fa05800b330 comp rx=0 tx=0).stop 2023-08-11T15:48:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 -- 172.21.15.32:0/2898246017 shutdown_connections 2023-08-11T15:48:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 --2- 172.21.15.32:0/2898246017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa060101270 0x7fa06010d780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 --2- 172.21.15.32:0/2898246017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0601068c0 0x7fa060106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 --2- 172.21.15.32:0/2898246017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0601008b0 0x7fa060100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 -- 172.21.15.32:0/2898246017 >> 172.21.15.32:0/2898246017 conn(0x7fa0600fc380 msgr2=0x7fa0600fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 -- 172.21.15.32:0/2898246017 shutdown_connections 2023-08-11T15:48:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.020+0000 7fa0657ff700 1 -- 172.21.15.32:0/2898246017 wait complete. 2023-08-11T15:48:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.021+0000 7fa0657ff700 1 Processor -- start 2023-08-11T15:48:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.021+0000 7fa0657ff700 1 -- start start 2023-08-11T15:48:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa0657ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0601008b0 0x7fa06019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa0657ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060101270 0x7fa06019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa0657ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0601068c0 0x7fa0601989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa0657ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060199020 con 0x7fa060101270 2023-08-11T15:48:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa0657ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060199190 con 0x7fa0601068c0 2023-08-11T15:48:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa0657ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060199350 con 0x7fa0601008b0 2023-08-11T15:48:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060101270 0x7fa06019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060101270 0x7fa06019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51700/0 (socket says 172.21.15.32:51700) 2023-08-11T15:48:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3763203866 learned_addr learned my addr 172.21.15.32:0/3763203866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0601068c0 0x7fa0601989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05effd700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0601008b0 0x7fa06019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3763203866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0601008b0 msgr2=0x7fa06019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0601008b0 0x7fa06019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3763203866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0601068c0 msgr2=0x7fa0601989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0601068c0 0x7fa0601989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3763203866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058009c40 con 0x7fa060101270 2023-08-11T15:48:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.022+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0601068c0 0x7fa0601989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.023+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060101270 0x7fa06019f090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e9a0 tx=0x7fa05400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.023+0000 7fa047fff700 1 -- 172.21.15.32:0/3763203866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054009790 con 0x7fa060101270 2023-08-11T15:48:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.023+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa060199630 con 0x7fa060101270 2023-08-11T15:48:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.023+0000 7fa047fff700 1 -- 172.21.15.32:0/3763203866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa05400ce80 con 0x7fa060101270 2023-08-11T15:48:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.023+0000 7fa047fff700 1 -- 172.21.15.32:0/3763203866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054018680 con 0x7fa060101270 2023-08-11T15:48:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.023+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0601a5b40 con 0x7fa060101270 2023-08-11T15:48:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.027+0000 7fa047fff700 1 -- 172.21.15.32:0/3763203866 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa054010430 con 0x7fa060101270 2023-08-11T15:48:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.028+0000 7fa0791deb80 1 Processor -- start 2023-08-11T15:48:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa047fff700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa0480a0140 0x7fa0480a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa0791deb80 1 -- start start 2023-08-11T15:48:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa047fff700 1 -- 172.21.15.32:0/3763203866 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa054014070 con 0x7fa060101270 2023-08-11T15:48:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa0480a0140 0x7fa0480a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27df260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa0791deb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f27325d0 con 0x5619f27d9ff0 2023-08-11T15:48:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa0791deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f272fef0 con 0x5619f27d9630 2023-08-11T15:48:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa0791deb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f272b7e0 con 0x5619f27dee80 2023-08-11T15:48:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.029+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27d9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35178/0 (socket says 172.21.15.32:35178) 2023-08-11T15:48:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 -- 172.21.15.32:0/693250663 learned_addr learned my addr 172.21.15.32:0/693250663 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05effd700 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 -- 172.21.15.32:0/693250663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 msgr2=0x5619f27df260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27df260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 -- 172.21.15.32:0/693250663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 msgr2=0x5619f27e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 -- 172.21.15.32:0/693250663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f25ab5d0 con 0x5619f27d9630 2023-08-11T15:48:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:48:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa0480a0140 0x7fa0480a2600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa06019f850 tx=0x7fa058009700 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27d9ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0540a0780 tx=0x7fa054098b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f578e161d2b8ff server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa0457fa700 1 -- 172.21.15.32:0/693250663 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05409ccd0 con 0x5619f27d9630 2023-08-11T15:48:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa0457fa700 1 -- 172.21.15.32:0/693250663 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0540b6e90 con 0x5619f27d9630 2023-08-11T15:48:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.030+0000 7fa0457fa700 1 -- 172.21.15.32:0/693250663 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054098390 con 0x5619f27d9630 2023-08-11T15:48:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 -- 172.21.15.32:0/693250663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 msgr2=0x5619f27d9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27d9ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0540a0780 tx=0x7fa054098b10 comp rx=0 tx=0).stop 2023-08-11T15:48:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 -- 172.21.15.32:0/693250663 shutdown_connections 2023-08-11T15:48:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27df260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 --2- 172.21.15.32:0/693250663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27d9ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 -- 172.21.15.32:0/693250663 >> 172.21.15.32:0/693250663 conn(0x5619f27068f0 msgr2=0x5619f27d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.031+0000 7fa0791deb80 1 -- 172.21.15.32:0/693250663 shutdown_connections 2023-08-11T15:48:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.032+0000 7fa0791deb80 1 -- 172.21.15.32:0/693250663 wait complete. 2023-08-11T15:48:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.032+0000 7fa0791deb80 1 -- start start 2023-08-11T15:48:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.032+0000 7fa0791deb80 1 Processor -- start 2023-08-11T15:48:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa0791deb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f26c9a60 con 0x5619f27d9ff0 2023-08-11T15:48:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa0791deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f26c9760 con 0x5619f27d9630 2023-08-11T15:48:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa0791deb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f25ad8c0 con 0x5619f27dee80 2023-08-11T15:48:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa0791deb80 10 client.0 _ll_register_callbacks cb 0x5619f26cca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35182/0 (socket says 172.21.15.32:35182) 2023-08-11T15:48:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 -- 172.21.15.32:0/95355720 learned_addr learned my addr 172.21.15.32:0/95355720 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 -- 172.21.15.32:0/95355720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 msgr2=0x5619f27e2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27e2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 -- 172.21.15.32:0/95355720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 msgr2=0x5619f27e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05effd700 1 -- 172.21.15.32:0/95355720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0601a5b40 con 0x5619f27d9630 2023-08-11T15:48:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.033+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:48:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa05effd700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e8af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa050012840 tx=0x7fa050012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b90cf2971e07933 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa042ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:48:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa042ff5700 1 -- 172.21.15.32:0/95355720 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050017070 con 0x5619f27d9630 2023-08-11T15:48:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619f27cc7e0 con 0x5619f27d9630 2023-08-11T15:48:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa042ff5700 1 -- 172.21.15.32:0/95355720 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa05000f540 con 0x5619f27d9630 2023-08-11T15:48:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa042ff5700 1 -- 172.21.15.32:0/95355720 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0500122e0 con 0x5619f27d9630 2023-08-11T15:48:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa0791deb80 10 client.25562 fetch_fsmap learned FSMap version 25 2023-08-11T15:48:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.034+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619f2557d80 con 0x5619f27d9630 2023-08-11T15:48:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa042ff5700 1 -- 172.21.15.32:0/95355720 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619f27cc7e0 con 0x5619f27d9630 2023-08-11T15:48:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa042ff5700 1 -- 172.21.15.32:0/95355720 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa050010950 con 0x5619f27d9630 2023-08-11T15:48:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa0791deb80 10 client.25562 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:48:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa0791deb80 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa0791deb80 20 client.25562 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa0791deb80 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f27f3020 0x5619f27f5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa0791deb80 4 client.25562 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:48:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619f27a8480 con 0x5619f27f3020 2023-08-11T15:48:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.035+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f27f3020 0x5619f27f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.036+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f27f3020 0x5619f27f5440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27d13bbf9439c35 server_cookie=ab9e9d4fc65be4b5 in_seq=0 out_seq=0 2023-08-11T15:48:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.036+0000 7fa042ff5700 10 client.25562 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa042ff5700 1 -- 172.21.15.32:0/95355720 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5619f27a8480 con 0x5619f27f3020 2023-08-11T15:48:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa042ff5700 10 client.25562 handle_command_reply: tid=0 2023-08-11T15:48:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 client.25562 shutdown 2023-08-11T15:48:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 msgr2=0x5619f27e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e8af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa050012840 tx=0x7fa050012b50 comp rx=0 tx=0).stop 2023-08-11T15:48:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 shutdown_connections 2023-08-11T15:48:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f27f3020 0x5619f27f5440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa042ff5700 0 client.25562 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e8af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 --2- 172.21.15.32:0/95355720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 >> 172.21.15.32:0/95355720 conn(0x5619f27068f0 msgr2=0x5619f27d35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 shutdown_connections 2023-08-11T15:48:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.038+0000 7fa0791deb80 1 -- 172.21.15.32:0/95355720 wait complete. 2023-08-11T15:48:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.039+0000 7fa0791deb80 20 client.25562 trim_cache size 0 max 16384 2023-08-11T15:48:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.057+0000 7fa0791deb80 1 Processor -- start 2023-08-11T15:48:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.057+0000 7fa0791deb80 1 -- start start 2023-08-11T15:48:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa0791deb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f27d55a0 con 0x5619f27d9ff0 2023-08-11T15:48:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa0791deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f27d5db0 con 0x5619f27d9630 2023-08-11T15:48:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa0791deb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f27ddaa0 con 0x5619f27dee80 2023-08-11T15:48:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35188/0 (socket says 172.21.15.32:35188) 2023-08-11T15:48:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 -- 172.21.15.32:0/1671030154 learned_addr learned my addr 172.21.15.32:0/1671030154 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 -- 172.21.15.32:0/1671030154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 msgr2=0x5619f27d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 -- 172.21.15.32:0/1671030154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 msgr2=0x5619f27e8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05effd700 1 -- 172.21.15.32:0/1671030154 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f2540c00 con 0x5619f27d9630 2023-08-11T15:48:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.058+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.059+0000 7fa05effd700 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e09d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa050010d80 tx=0x7fa050012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2b53ceee69eaee server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.059+0000 7fa045ffb700 1 -- 172.21.15.32:0/1671030154 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050017070 con 0x5619f27d9630 2023-08-11T15:48:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.059+0000 7fa045ffb700 1 -- 172.21.15.32:0/1671030154 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa050018040 con 0x5619f27d9630 2023-08-11T15:48:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.059+0000 7fa045ffb700 1 -- 172.21.15.32:0/1671030154 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050003680 con 0x5619f27d9630 2023-08-11T15:48:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 -- 172.21.15.32:0/1671030154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 msgr2=0x5619f27e09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e09d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa050010d80 tx=0x7fa050012000 comp rx=0 tx=0).stop 2023-08-11T15:48:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 -- 172.21.15.32:0/1671030154 shutdown_connections 2023-08-11T15:48:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27dee80 0x5619f27d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9630 0x5619f27e09d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 --2- 172.21.15.32:0/1671030154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27d9ff0 0x5619f27e8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 -- 172.21.15.32:0/1671030154 >> 172.21.15.32:0/1671030154 conn(0x5619f27068f0 msgr2=0x5619f27dcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 -- 172.21.15.32:0/1671030154 shutdown_connections 2023-08-11T15:48:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 -- 172.21.15.32:0/1671030154 wait complete. 2023-08-11T15:48:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.060+0000 7fa0791deb80 1 -- start start 2023-08-11T15:48:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 1 Processor -- start 2023-08-11T15:48:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27d9630 0x5619f283ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9ff0 0x5619f283b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27dee80 0x5619f283fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f28401b0 con 0x5619f27dee80 2023-08-11T15:48:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f2840320 con 0x5619f27d9ff0 2023-08-11T15:48:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f28404e0 con 0x5619f27d9630 2023-08-11T15:48:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa0791deb80 10 client.0 _ll_register_callbacks cb 0x7fa050009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9ff0 0x5619f283b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa05e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9ff0 0x5619f283b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:35196/0 (socket says 172.21.15.32:35196) 2023-08-11T15:48:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3530398752 learned_addr learned my addr 172.21.15.32:0/3530398752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa05effd700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27d9630 0x5619f283ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.061+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27dee80 0x5619f283fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3530398752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27d9630 msgr2=0x5619f283ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27d9630 0x5619f283ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3530398752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27dee80 msgr2=0x5619f283fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27dee80 0x5619f283fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa05e7fc700 1 -- 172.21.15.32:0/3530398752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619f25ab5d0 con 0x5619f27d9ff0 2023-08-11T15:48:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa05f7fe700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27dee80 0x5619f283fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:48:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa05e7fc700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9ff0 0x5619f283b350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c5b0 tx=0x7fa054010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ece4ac197e45a56 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa042ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:48:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa042ff5700 1 -- 172.21.15.32:0/3530398752 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05409ccd0 con 0x5619f27d9ff0 2023-08-11T15:48:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa042ff5700 1 -- 172.21.15.32:0/3530398752 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa054099100 con 0x5619f27d9ff0 2023-08-11T15:48:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0540b6e90 con 0x5619f27d9ff0 2023-08-11T15:48:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa042ff5700 1 -- 172.21.15.32:0/3530398752 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0540992c0 con 0x5619f27d9ff0 2023-08-11T15:48:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa0791deb80 10 client.25568 fetch_fsmap learned FSMap version 25 2023-08-11T15:48:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.062+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619f2557d80 con 0x5619f27d9ff0 2023-08-11T15:48:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa042ff5700 1 -- 172.21.15.32:0/3530398752 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0540b6e90 con 0x5619f27d9ff0 2023-08-11T15:48:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa042ff5700 1 -- 172.21.15.32:0/3530398752 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa0540b8140 con 0x5619f27d9ff0 2023-08-11T15:48:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa0791deb80 10 client.25568 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:48:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa0791deb80 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa0791deb80 20 client.25568 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa0791deb80 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f2847a40 0x5619f2849e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa0791deb80 4 client.25568 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:48:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619f27247c0 con 0x5619f2847a40 2023-08-11T15:48:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.063+0000 7fa05effd700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f2847a40 0x5619f2849e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.064+0000 7fa042ff5700 10 client.25568 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.064+0000 7fa05effd700 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f2847a40 0x5619f2849e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e19d1cd0a00a2a server_cookie=538530494c36ee04 in_seq=0 out_seq=0 2023-08-11T15:48:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.321+0000 7fa042ff5700 1 -- 172.21.15.32:0/3530398752 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619f27247c0 con 0x5619f2847a40 2023-08-11T15:48:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.321+0000 7fa042ff5700 10 client.25568 handle_command_reply: tid=0 2023-08-11T15:48:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.321+0000 7fa0791deb80 1 client.25568 shutdown 2023-08-11T15:48:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.321+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9ff0 msgr2=0x5619f283b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.321+0000 7fa0791deb80 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9ff0 0x5619f283b350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c5b0 tx=0x7fa054010b10 comp rx=0 tx=0).stop 2023-08-11T15:48:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 shutdown_connections 2023-08-11T15:48:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5619f2847a40 0x5619f2849e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619f27dee80 0x5619f283fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619f27d9630 0x5619f283ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 --2- 172.21.15.32:0/3530398752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5619f27d9ff0 0x5619f283b350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 >> 172.21.15.32:0/3530398752 conn(0x5619f27068f0 msgr2=0x5619f27d8620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa042ff5700 0 client.25568 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 shutdown_connections 2023-08-11T15:48:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 1 -- 172.21.15.32:0/3530398752 wait complete. 2023-08-11T15:48:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.322+0000 7fa0791deb80 20 client.25568 trim_cache size 0 max 16384 2023-08-11T15:48:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:48:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:48:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:48:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:48:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa0480a0140 msgr2=0x7fa0480a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa0480a0140 0x7fa0480a2600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa06019f850 tx=0x7fa058009700 comp rx=0 tx=0).stop 2023-08-11T15:48:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060101270 msgr2=0x7fa06019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060101270 0x7fa06019f090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e9a0 tx=0x7fa05400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 shutdown_connections 2023-08-11T15:48:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa0480a0140 0x7fa0480a2600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0601068c0 0x7fa0601989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0601008b0 0x7fa06019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 --2- 172.21.15.32:0/3763203866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060101270 0x7fa06019f090 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 >> 172.21.15.32:0/3763203866 conn(0x7fa0600fc380 msgr2=0x7fa06010bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 shutdown_connections 2023-08-11T15:48:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:44.323+0000 7fa0657ff700 1 -- 172.21.15.32:0/3763203866 wait complete. 2023-08-11T15:48:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:44 smithi032 ceph-mon[97103]: pgmap v66: 129 pgs: 129 active+clean; 49 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2023-08-11T15:48:44.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:44 smithi192 ceph-mon[130806]: pgmap v66: 129 pgs: 129 active+clean; 49 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2023-08-11T15:48:44.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:44 smithi134 ceph-mon[130447]: pgmap v66: 129 pgs: 129 active+clean; 49 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2023-08-11T15:48:44.737 INFO:tasks.workunit.client.0.smithi032.stdout: 1 152297 13.98 MB/sec warmup 60 sec latency 196.646 ms 2023-08-11T15:48:45.739 INFO:tasks.workunit.client.0.smithi032.stdout: 1 154896 13.94 MB/sec warmup 61 sec latency 22.364 ms 2023-08-11T15:48:46.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:46 smithi032 ceph-mon[97103]: pgmap v67: 129 pgs: 129 active+clean; 49 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 174 op/s 2023-08-11T15:48:46.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:46 smithi192 ceph-mon[130806]: pgmap v67: 129 pgs: 129 active+clean; 49 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 174 op/s 2023-08-11T15:48:46.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:46 smithi134 ceph-mon[130447]: pgmap v67: 129 pgs: 129 active+clean; 49 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 174 op/s 2023-08-11T15:48:46.739 INFO:tasks.workunit.client.0.smithi032.stdout: 1 156494 13.91 MB/sec warmup 62 sec latency 140.798 ms 2023-08-11T15:48:47.739 INFO:tasks.workunit.client.0.smithi032.stdout: 1 157702 13.77 MB/sec warmup 63 sec latency 237.492 ms 2023-08-11T15:48:48.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:47 smithi134 ceph-mon[130447]: pgmap v68: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 478 op/s 2023-08-11T15:48:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:47 smithi032 ceph-mon[97103]: pgmap v68: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 478 op/s 2023-08-11T15:48:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:47 smithi192 ceph-mon[130806]: pgmap v68: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 478 op/s 2023-08-11T15:48:48.739 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160064 13.77 MB/sec warmup 64 sec latency 27.843 ms 2023-08-11T15:48:49.739 INFO:tasks.workunit.client.0.smithi032.stdout: 1 163263 13.83 MB/sec warmup 65 sec latency 6.310 ms 2023-08-11T15:48:50.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:50 smithi032 ceph-mon[97103]: pgmap v69: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 449 op/s 2023-08-11T15:48:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:50 smithi192 ceph-mon[130806]: pgmap v69: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 449 op/s 2023-08-11T15:48:50.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:50 smithi134 ceph-mon[130447]: pgmap v69: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 449 op/s 2023-08-11T15:48:50.739 INFO:tasks.workunit.client.0.smithi032.stdout: 1 165651 13.76 MB/sec warmup 66 sec latency 19.101 ms 2023-08-11T15:48:51.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:51 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:48:51.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:51 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:48:51.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:51 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:48:51.741 INFO:tasks.workunit.client.0.smithi032.stdout: 1 167399 13.73 MB/sec warmup 67 sec latency 128.526 ms 2023-08-11T15:48:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: pgmap v70: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 384 op/s 2023-08-11T15:48:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:48:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:48:52.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:52 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: pgmap v70: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 384 op/s 2023-08-11T15:48:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:48:52.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:48:52.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:52 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: pgmap v70: 129 pgs: 129 active+clean; 85 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 384 op/s 2023-08-11T15:48:52.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:48:52.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:48:52.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:52 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:48:52.742 INFO:tasks.workunit.client.0.smithi032.stdout: 1 169291 13.64 MB/sec warmup 68 sec latency 195.859 ms 2023-08-11T15:48:53.742 INFO:tasks.workunit.client.0.smithi032.stdout: 1 171262 13.62 MB/sec warmup 69 sec latency 20.238 ms 2023-08-11T15:48:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:54 smithi032 ceph-mon[97103]: pgmap v71: 129 pgs: 129 active+clean; 69 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 624 op/s 2023-08-11T15:48:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:54 smithi192 ceph-mon[130806]: pgmap v71: 129 pgs: 129 active+clean; 69 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 624 op/s 2023-08-11T15:48:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:54 smithi134 ceph-mon[130447]: pgmap v71: 129 pgs: 129 active+clean; 69 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 624 op/s 2023-08-11T15:48:54.742 INFO:tasks.workunit.client.0.smithi032.stdout: 1 174237 13.68 MB/sec warmup 70 sec latency 22.079 ms 2023-08-11T15:48:55.742 INFO:tasks.workunit.client.0.smithi032.stdout: 1 177190 13.70 MB/sec warmup 71 sec latency 17.286 ms 2023-08-11T15:48:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:56 smithi032 ceph-mon[97103]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.3 MiB/s wr, 544 op/s 2023-08-11T15:48:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:56 smithi192 ceph-mon[130806]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.3 MiB/s wr, 544 op/s 2023-08-11T15:48:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:56 smithi134 ceph-mon[130447]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.3 MiB/s wr, 544 op/s 2023-08-11T15:48:56.742 INFO:tasks.workunit.client.0.smithi032.stdout: 1 179345 13.65 MB/sec warmup 72 sec latency 12.391 ms 2023-08-11T15:48:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 -- 172.21.15.32:0/3452025936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79441068b0 msgr2=0x7f7944106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 --2- 172.21.15.32:0/3452025936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79441068b0 0x7f7944106c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7934009990 tx=0x7f7934009ca0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 -- 172.21.15.32:0/3452025936 shutdown_connections 2023-08-11T15:48:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 --2- 172.21.15.32:0/3452025936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7944101260 0x7f794410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 --2- 172.21.15.32:0/3452025936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79441068b0 0x7f7944106c90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 --2- 172.21.15.32:0/3452025936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79441008a0 0x7f7944100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 -- 172.21.15.32:0/3452025936 >> 172.21.15.32:0/3452025936 conn(0x7f79440fc370 msgr2=0x7f79440fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 -- 172.21.15.32:0/3452025936 shutdown_connections 2023-08-11T15:48:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.484+0000 7f79493ff700 1 -- 172.21.15.32:0/3452025936 wait complete. 2023-08-11T15:48:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.486+0000 7f79493ff700 1 Processor -- start 2023-08-11T15:48:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.486+0000 7f79493ff700 1 -- start start 2023-08-11T15:48:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79493ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79441008a0 0x7f794419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79493ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7944101260 0x7f794419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79493ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79441068b0 0x7f79441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79493ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944199020 con 0x7f79441008a0 2023-08-11T15:48:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79493ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944199190 con 0x7f7944101260 2023-08-11T15:48:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79493ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944199350 con 0x7f79441068b0 2023-08-11T15:48:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79441008a0 0x7f794419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79427fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7944101260 0x7f794419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79441008a0 0x7f794419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47058/0 (socket says 172.21.15.32:47058) 2023-08-11T15:48:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f7942ffd700 1 -- 172.21.15.32:0/708831071 learned_addr learned my addr 172.21.15.32:0/708831071 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79437fe700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79441068b0 0x7f79441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.487+0000 7f79427fc700 1 -- 172.21.15.32:0/708831071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79441068b0 msgr2=0x7f79441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79427fc700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79441068b0 0x7f79441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79427fc700 1 -- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79441008a0 msgr2=0x7f794419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79427fc700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79441008a0 0x7f794419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79427fc700 1 -- 172.21.15.32:0/708831071 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340095f0 con 0x7f7944101260 2023-08-11T15:48:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f7942ffd700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79441008a0 0x7f794419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79437fe700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79441068b0 0x7f79441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79427fc700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7944101260 0x7f794419f090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9f0 tx=0x7f793800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f792bfff700 1 -- 172.21.15.32:0/708831071 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800c9c0 con 0x7f7944101260 2023-08-11T15:48:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f792bfff700 1 -- 172.21.15.32:0/708831071 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7938010430 con 0x7f7944101260 2023-08-11T15:48:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7944199630 con 0x7f7944101260 2023-08-11T15:48:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f792bfff700 1 -- 172.21.15.32:0/708831071 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938018680 con 0x7f7944101260 2023-08-11T15:48:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.488+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79441a5b40 con 0x7f7944101260 2023-08-11T15:48:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 Processor -- start 2023-08-11T15:48:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 -- start start 2023-08-11T15:48:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd79feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a0ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd79575d0 con 0x560bd79feff0 2023-08-11T15:48:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd7954ef0 con 0x560bd7a03e80 2023-08-11T15:48:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.493+0000 7f795cc61b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd79507e0 con 0x560bd79fe630 2023-08-11T15:48:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f792bfff700 1 -- 172.21.15.32:0/708831071 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f7938010860 con 0x7f7944101260 2023-08-11T15:48:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f79427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a0ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd79feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f79427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a0ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47072/0 (socket says 172.21.15.32:47072) 2023-08-11T15:48:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f79427fc700 1 -- 172.21.15.32:0/2334264004 learned_addr learned my addr 172.21.15.32:0/2334264004 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f7942ffd700 1 -- 172.21.15.32:0/2334264004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 msgr2=0x560bd7a04260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f7942ffd700 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f7942ffd700 1 -- 172.21.15.32:0/2334264004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 msgr2=0x560bd7a0ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f7942ffd700 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a0ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.494+0000 7f7942ffd700 1 -- 172.21.15.32:0/2334264004 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bd77d05d0 con 0x560bd79fe630 2023-08-11T15:48:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.495+0000 7f79427fc700 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a0ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:48:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.495+0000 7f7942ffd700 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd79feab0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7934005910 tx=0x7f793400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c440e2876eafe9d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.495+0000 7f79297fa700 1 -- 172.21.15.32:0/2334264004 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793401d050 con 0x560bd79fe630 2023-08-11T15:48:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.495+0000 7f79297fa700 1 -- 172.21.15.32:0/2334264004 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793401e040 con 0x560bd79fe630 2023-08-11T15:48:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.495+0000 7f79297fa700 1 -- 172.21.15.32:0/2334264004 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400f7a0 con 0x560bd79fe630 2023-08-11T15:48:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 -- 172.21.15.32:0/2334264004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 msgr2=0x560bd79feab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd79feab0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7934005910 tx=0x7f793400b610 comp rx=0 tx=0).stop 2023-08-11T15:48:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 -- 172.21.15.32:0/2334264004 shutdown_connections 2023-08-11T15:48:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a0ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 --2- 172.21.15.32:0/2334264004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd79feab0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 -- 172.21.15.32:0/2334264004 >> 172.21.15.32:0/2334264004 conn(0x560bd792b8f0 msgr2=0x560bd79f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 -- 172.21.15.32:0/2334264004 shutdown_connections 2023-08-11T15:48:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f795cc61b80 1 -- 172.21.15.32:0/2334264004 wait complete. 2023-08-11T15:48:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.496+0000 7f792bfff700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f792c0a00d0 0x7f792c0a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.497+0000 7f792bfff700 1 -- 172.21.15.32:0/708831071 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f7938014070 con 0x7f7944101260 2023-08-11T15:48:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.497+0000 7f795cc61b80 1 -- start start 2023-08-11T15:48:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.497+0000 7f79437fe700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f792c0a00d0 0x7f792c0a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.497+0000 7f795cc61b80 1 Processor -- start 2023-08-11T15:48:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f79437fe700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f792c0a00d0 0x7f792c0a2590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f794419f850 tx=0x7f793c004ad0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd79feff0 0x560bd7a0de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd7a03e80 0x560bd7a07c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f795cc61b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd78eea60 con 0x560bd79fe630 2023-08-11T15:48:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f795cc61b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd78ee760 con 0x560bd79feff0 2023-08-11T15:48:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f795cc61b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd77d28c0 con 0x560bd7a03e80 2023-08-11T15:48:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f795cc61b80 10 client.0 _ll_register_callbacks cb 0x560bd78f1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f79427fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd79feff0 0x560bd7a0de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f79437fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd7a03e80 0x560bd7a07c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f79427fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd79feff0 0x560bd7a0de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:57884/0 (socket says 172.21.15.32:57884) 2023-08-11T15:48:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.498+0000 7f79427fc700 1 -- 172.21.15.32:0/541625061 learned_addr learned my addr 172.21.15.32:0/541625061 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f7942ffd700 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f79437fe700 1 -- 172.21.15.32:0/541625061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd79feff0 msgr2=0x560bd7a0de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f79437fe700 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd79feff0 0x560bd7a0de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f79437fe700 1 -- 172.21.15.32:0/541625061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 msgr2=0x560bd7a0d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f79437fe700 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f79437fe700 1 -- 172.21.15.32:0/541625061 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794410e070 con 0x560bd7a03e80 2023-08-11T15:48:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f7942ffd700 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:48:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f79437fe700 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd7a03e80 0x560bd7a07c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f793c007fd0 tx=0x7f793c009130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2ca2fd324f334e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f7926ff5700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f7926ff5700 1 -- 172.21.15.32:0/541625061 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c015e10 con 0x560bd7a03e80 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bd79f17e0 con 0x560bd7a03e80 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.499+0000 7f7926ff5700 1 -- 172.21.15.32:0/541625061 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f793c021440 con 0x560bd7a03e80 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f7926ff5700 1 -- 172.21.15.32:0/541625061 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f793c021600 con 0x560bd7a03e80 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 10 client.25579 fetch_fsmap learned FSMap version 25 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560bd777cd80 con 0x560bd7a03e80 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f7926ff5700 1 -- 172.21.15.32:0/541625061 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560bd79f17e0 con 0x560bd7a03e80 2023-08-11T15:48:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f7926ff5700 1 -- 172.21.15.32:0/541625061 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f793c02a800 con 0x560bd7a03e80 2023-08-11T15:48:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 10 client.25579 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:48:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 20 client.25579 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a17e40 0x560bd7a1a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 4 client.25579 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:48:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.500+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bd79cd480 con 0x560bd7a17e40 2023-08-11T15:48:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.501+0000 7f7942ffd700 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a17e40 0x560bd7a1a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.502+0000 7f7942ffd700 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a17e40 0x560bd7a1a260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99460569233eea5 server_cookie=dd6ce7653be74775 in_seq=0 out_seq=0 2023-08-11T15:48:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.502+0000 7f7926ff5700 10 client.25579 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.502+0000 7f7926ff5700 1 -- 172.21.15.32:0/541625061 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x560bd79cd480 con 0x560bd7a17e40 2023-08-11T15:48:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.502+0000 7f7926ff5700 10 client.25579 handle_command_reply: tid=0 2023-08-11T15:48:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 client.25579 shutdown 2023-08-11T15:48:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd7a03e80 msgr2=0x560bd7a07c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd7a03e80 0x560bd7a07c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f793c007fd0 tx=0x7f793c009130 comp rx=0 tx=0).stop 2023-08-11T15:48:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 shutdown_connections 2023-08-11T15:48:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a17e40 0x560bd7a1a260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd7a03e80 0x560bd7a07c80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 --2- 172.21.15.32:0/541625061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd79feff0 0x560bd7a0de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f7926ff5700 0 client.25579 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 >> 172.21.15.32:0/541625061 conn(0x560bd792b8f0 msgr2=0x560bd79f85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 shutdown_connections 2023-08-11T15:48:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 1 -- 172.21.15.32:0/541625061 wait complete. 2023-08-11T15:48:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.503+0000 7f795cc61b80 20 client.25579 trim_cache size 0 max 16384 2023-08-11T15:48:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.519+0000 7f795cc61b80 1 Processor -- start 2023-08-11T15:48:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.519+0000 7f795cc61b80 1 -- start start 2023-08-11T15:48:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd7a1ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd79f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f795cc61b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd7a08b50 con 0x560bd79feff0 2023-08-11T15:48:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f795cc61b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd79fa5a0 con 0x560bd7a03e80 2023-08-11T15:48:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f795cc61b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd79f1620 con 0x560bd79fe630 2023-08-11T15:48:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd7a1ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f79427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd7a1ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:59580/0 (socket says 172.21.15.32:59580) 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 -- 172.21.15.32:0/1987198969 learned_addr learned my addr 172.21.15.32:0/1987198969 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f79437fe700 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd79f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 -- 172.21.15.32:0/1987198969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 msgr2=0x560bd79f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd79f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 -- 172.21.15.32:0/1987198969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 msgr2=0x560bd7a03600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a03600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.520+0000 7f7942ffd700 1 -- 172.21.15.32:0/1987198969 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bd7765c00 con 0x560bd79fe630 2023-08-11T15:48:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.521+0000 7f7942ffd700 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd7a1ebe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7934023ee0 tx=0x7f793401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b5a2bbeec74b61a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.521+0000 7f7929ffb700 1 -- 172.21.15.32:0/1987198969 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793401d070 con 0x560bd79fe630 2023-08-11T15:48:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.521+0000 7f7929ffb700 1 -- 172.21.15.32:0/1987198969 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7934004830 con 0x560bd79fe630 2023-08-11T15:48:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.521+0000 7f7929ffb700 1 -- 172.21.15.32:0/1987198969 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934005280 con 0x560bd79fe630 2023-08-11T15:48:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 -- 172.21.15.32:0/1987198969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 msgr2=0x560bd7a1ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd7a1ebe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7934023ee0 tx=0x7f793401a040 comp rx=0 tx=0).stop 2023-08-11T15:48:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 -- 172.21.15.32:0/1987198969 shutdown_connections 2023-08-11T15:48:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd79f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79fe630 0x560bd7a1ebe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 --2- 172.21.15.32:0/1987198969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79feff0 0x560bd7a03600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 -- 172.21.15.32:0/1987198969 >> 172.21.15.32:0/1987198969 conn(0x560bd792b8f0 msgr2=0x560bd7a02ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 -- 172.21.15.32:0/1987198969 shutdown_connections 2023-08-11T15:48:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 -- 172.21.15.32:0/1987198969 wait complete. 2023-08-11T15:48:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.522+0000 7f795cc61b80 1 -- start start 2023-08-11T15:48:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 1 Processor -- start 2023-08-11T15:48:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79feff0 0x560bd7a0b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a6ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd7a0bff0 con 0x560bd79fe630 2023-08-11T15:48:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd7a0c160 con 0x560bd7a03e80 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd7a6b050 con 0x560bd79feff0 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f795cc61b80 10 client.0 _ll_register_callbacks cb 0x7f793c0157e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f79437fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a6ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47098/0 (socket says 172.21.15.32:47098) 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.523+0000 7f7942ffd700 1 -- 172.21.15.32:0/3609554138 learned_addr learned my addr 172.21.15.32:0/3609554138 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f79427fc700 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79feff0 0x560bd7a0b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7942ffd700 1 -- 172.21.15.32:0/3609554138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79feff0 msgr2=0x560bd7a0b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7942ffd700 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79feff0 0x560bd7a0b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7942ffd700 1 -- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 msgr2=0x560bd7a6ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:48:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7942ffd700 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a6ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7942ffd700 1 -- 172.21.15.32:0/3609554138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bd77d05d0 con 0x560bd79fe630 2023-08-11T15:48:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f79437fe700 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a6ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:48:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7942ffd700 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0b320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7934005cf0 tx=0x7f7934023f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aba4b5f0e6bc387 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:48:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7926ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7926ff5700 1 -- 172.21.15.32:0/3609554138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793401d050 con 0x560bd79fe630 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f793401e040 con 0x560bd79fe630 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.524+0000 7f7926ff5700 1 -- 172.21.15.32:0/3609554138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f7934033040 con 0x560bd79fe630 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f7926ff5700 1 -- 172.21.15.32:0/3609554138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7934004db0 con 0x560bd79fe630 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 10 client.15576 fetch_fsmap learned FSMap version 25 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560bd77a31f0 con 0x560bd79fe630 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f7926ff5700 1 -- 172.21.15.32:0/3609554138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793401e040 con 0x560bd79fe630 2023-08-11T15:48:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f7926ff5700 1 -- 172.21.15.32:0/3609554138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79340053d0 con 0x560bd79fe630 2023-08-11T15:48:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 10 client.15576 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:48:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:48:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 20 client.15576 populate_metadata read hostname 'smithi032' 2023-08-11T15:48:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a6c300 0x560bd7a6e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:48:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 4 client.15576 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:48:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.525+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560bd79497c0 con 0x560bd7a6c300 2023-08-11T15:48:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.526+0000 7f79427fc700 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a6c300 0x560bd7a6e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:48:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.526+0000 7f79427fc700 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a6c300 0x560bd7a6e720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f05305f3f56d68 server_cookie=613ca18922317896 in_seq=0 out_seq=0 2023-08-11T15:48:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.526+0000 7f7926ff5700 10 client.15576 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:57.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 181448 13.65 MB/sec warmup 73 sec latency 37.635 ms 2023-08-11T15:48:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.755+0000 7f7926ff5700 1 -- 172.21.15.32:0/3609554138 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560bd79497c0 con 0x560bd7a6c300 2023-08-11T15:48:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f7926ff5700 10 client.15576 handle_command_reply: tid=0 2023-08-11T15:48:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 client.15576 shutdown 2023-08-11T15:48:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 msgr2=0x560bd7a0b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0b320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7934005cf0 tx=0x7f7934023f10 comp rx=0 tx=0).stop 2023-08-11T15:48:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 shutdown_connections 2023-08-11T15:48:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x560bd7a6c300 0x560bd7a6e720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560bd79fe630 0x560bd7a0b320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bd7a03e80 0x560bd7a6ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 --2- 172.21.15.32:0/3609554138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560bd79feff0 0x560bd7a0b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 >> 172.21.15.32:0/3609554138 conn(0x560bd792b8f0 msgr2=0x560bd79fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f7926ff5700 0 client.15576 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:48:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 shutdown_connections 2023-08-11T15:48:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.756+0000 7f795cc61b80 1 -- 172.21.15.32:0/3609554138 wait complete. 2023-08-11T15:48:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.757+0000 7f795cc61b80 20 client.15576 trim_cache size 0 max 16384 2023-08-11T15:48:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:48:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:48:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:48:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:48:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f792c0a00d0 msgr2=0x7f792c0a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f792c0a00d0 0x7f792c0a2590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f794419f850 tx=0x7f793c004ad0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7944101260 msgr2=0x7f794419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7944101260 0x7f794419f090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9f0 tx=0x7f793800ed00 comp rx=0 tx=0).stop 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 shutdown_connections 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f792c0a00d0 0x7f792c0a2590 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79441008a0 0x7f794419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79441068b0 0x7f79441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 --2- 172.21.15.32:0/708831071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7944101260 0x7f794419f090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 >> 172.21.15.32:0/708831071 conn(0x7f79440fc370 msgr2=0x7f794410bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:48:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 shutdown_connections 2023-08-11T15:48:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:48:57.758+0000 7f79493ff700 1 -- 172.21.15.32:0/708831071 wait complete. 2023-08-11T15:48:58.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:48:57 smithi134 ceph-mon[130447]: pgmap v73: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 693 op/s 2023-08-11T15:48:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:48:57 smithi032 ceph-mon[97103]: pgmap v73: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 693 op/s 2023-08-11T15:48:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:48:57 smithi192 ceph-mon[130806]: pgmap v73: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 693 op/s 2023-08-11T15:48:58.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 184535 13.70 MB/sec warmup 74 sec latency 15.177 ms 2023-08-11T15:48:59.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 186350 13.61 MB/sec warmup 75 sec latency 89.338 ms 2023-08-11T15:49:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:00 smithi032 ceph-mon[97103]: pgmap v74: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 389 op/s 2023-08-11T15:49:00.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:00 smithi192 ceph-mon[130806]: pgmap v74: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 389 op/s 2023-08-11T15:49:00.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:00 smithi134 ceph-mon[130447]: pgmap v74: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 389 op/s 2023-08-11T15:49:00.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 188563 13.61 MB/sec warmup 76 sec latency 124.056 ms 2023-08-11T15:49:01.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 189040 13.46 MB/sec warmup 77 sec latency 62.687 ms 2023-08-11T15:49:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:02 smithi032 ceph-mon[97103]: pgmap v75: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 389 op/s 2023-08-11T15:49:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:02 smithi192 ceph-mon[130806]: pgmap v75: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 389 op/s 2023-08-11T15:49:02.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:02 smithi134 ceph-mon[130447]: pgmap v75: 129 pgs: 129 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 389 op/s 2023-08-11T15:49:02.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 191939 13.51 MB/sec warmup 78 sec latency 46.696 ms 2023-08-11T15:49:03.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 194787 13.51 MB/sec warmup 79 sec latency 16.142 ms 2023-08-11T15:49:04.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:04 smithi032 ceph-mon[97103]: pgmap v76: 129 pgs: 129 active+clean; 37 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 647 op/s 2023-08-11T15:49:04.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:04 smithi192 ceph-mon[130806]: pgmap v76: 129 pgs: 129 active+clean; 37 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 647 op/s 2023-08-11T15:49:04.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:04 smithi134 ceph-mon[130447]: pgmap v76: 129 pgs: 129 active+clean; 37 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 647 op/s 2023-08-11T15:49:04.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 197473 13.49 MB/sec warmup 80 sec latency 6.398 ms 2023-08-11T15:49:05.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 199441 13.48 MB/sec warmup 81 sec latency 74.875 ms 2023-08-11T15:49:06.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:06 smithi032 ceph-mon[97103]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 407 op/s 2023-08-11T15:49:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:06 smithi192 ceph-mon[130806]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 407 op/s 2023-08-11T15:49:06.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:06 smithi134 ceph-mon[130447]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 407 op/s 2023-08-11T15:49:06.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 202575 13.55 MB/sec warmup 82 sec latency 7.042 ms 2023-08-11T15:49:07.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 205362 13.54 MB/sec warmup 83 sec latency 30.559 ms 2023-08-11T15:49:08.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:07 smithi134 ceph-mon[130447]: pgmap v78: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 491 op/s 2023-08-11T15:49:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:07 smithi032 ceph-mon[97103]: pgmap v78: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 491 op/s 2023-08-11T15:49:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:07 smithi192 ceph-mon[130806]: pgmap v78: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 491 op/s 2023-08-11T15:49:08.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 208088 13.53 MB/sec warmup 84 sec latency 5.992 ms 2023-08-11T15:49:09.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 210337 13.54 MB/sec warmup 85 sec latency 10.922 ms 2023-08-11T15:49:10.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:10 smithi032 ceph-mon[97103]: pgmap v79: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 342 op/s 2023-08-11T15:49:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:10 smithi192 ceph-mon[130806]: pgmap v79: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 342 op/s 2023-08-11T15:49:10.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:10 smithi134 ceph-mon[130447]: pgmap v79: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 342 op/s 2023-08-11T15:49:10.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 213261 13.58 MB/sec warmup 86 sec latency 15.162 ms 2023-08-11T15:49:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 -- 172.21.15.32:0/2287035040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840686d0 msgr2=0x7f238410d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 --2- 172.21.15.32:0/2287035040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840686d0 0x7f238410d720 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f237c00b0a0 tx=0x7f237c00b3b0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 -- 172.21.15.32:0/2287035040 shutdown_connections 2023-08-11T15:49:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 --2- 172.21.15.32:0/2287035040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23841068d0 0x7f2384106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 --2- 172.21.15.32:0/2287035040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840686d0 0x7f238410d720 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 --2- 172.21.15.32:0/2287035040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2384067d10 0x7f2384068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 -- 172.21.15.32:0/2287035040 >> 172.21.15.32:0/2287035040 conn(0x7f2384075250 msgr2=0x7f2384075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 -- 172.21.15.32:0/2287035040 shutdown_connections 2023-08-11T15:49:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.915+0000 7f238b7ff700 1 -- 172.21.15.32:0/2287035040 wait complete. 2023-08-11T15:49:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.916+0000 7f238b7ff700 1 Processor -- start 2023-08-11T15:49:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.916+0000 7f238b7ff700 1 -- start start 2023-08-11T15:49:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f238b7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384067d10 0x7f23840ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f238b7ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23840686d0 0x7f2384100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f238b7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23841068d0 0x7f23841a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f238b7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841a4fc0 con 0x7f2384067d10 2023-08-11T15:49:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f238b7ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841a5100 con 0x7f23840686d0 2023-08-11T15:49:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f238b7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841a5270 con 0x7f23841068d0 2023-08-11T15:49:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23841068d0 0x7f23841a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2388d9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23840686d0 0x7f2384100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23841068d0 0x7f23841a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:59044/0 (socket says 172.21.15.32:59044) 2023-08-11T15:49:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f238959b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384067d10 0x7f23840ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 -- 172.21.15.32:0/1582478074 learned_addr learned my addr 172.21.15.32:0/1582478074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 -- 172.21.15.32:0/1582478074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23840686d0 msgr2=0x7f2384100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23840686d0 0x7f2384100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 -- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384067d10 msgr2=0x7f23840ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384067d10 0x7f23840ffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2389d9c700 1 -- 172.21.15.32:0/1582478074 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2378009520 con 0x7f23841068d0 2023-08-11T15:49:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.917+0000 7f2388d9a700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23840686d0 0x7f2384100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:49:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.918+0000 7f2389d9c700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23841068d0 0x7f23841a4a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f237c009fd0 tx=0x7f237c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.918+0000 7f23767fc700 1 -- 172.21.15.32:0/1582478074 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c00e040 con 0x7f23841068d0 2023-08-11T15:49:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.918+0000 7f23767fc700 1 -- 172.21.15.32:0/1582478074 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f237c004e70 con 0x7f23841068d0 2023-08-11T15:49:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.918+0000 7f23767fc700 1 -- 172.21.15.32:0/1582478074 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0052a0 con 0x7f23841068d0 2023-08-11T15:49:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.918+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c009d00 con 0x7f23841068d0 2023-08-11T15:49:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.918+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23841a5790 con 0x7f23841068d0 2023-08-11T15:49:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.922+0000 7f23767fc700 1 -- 172.21.15.32:0/1582478074 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f237c01d470 con 0x7f23841068d0 2023-08-11T15:49:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.923+0000 7f23767fc700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f23700a0140 0x7f23700a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.923+0000 7f239f16fb80 1 Processor -- start 2023-08-11T15:49:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.923+0000 7f23767fc700 1 -- 172.21.15.32:0/1582478074 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f237c0ccb50 con 0x7f23841068d0 2023-08-11T15:49:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.924+0000 7f238959b700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f23700a0140 0x7f23700a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.924+0000 7f239f16fb80 1 -- start start 2023-08-11T15:49:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96eee80 0x5603f96ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9630 0x5603f96e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f239f16fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f96425d0 con 0x5603f96e9ff0 2023-08-11T15:49:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f239f16fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f963fef0 con 0x5603f96eee80 2023-08-11T15:49:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f239f16fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f963b7e0 con 0x5603f96e9630 2023-08-11T15:49:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f238959b700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f23700a0140 0x7f23700a2600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2378009d70 tx=0x7f2378009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f238959b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96eee80 0x5603f96ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f238959b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96eee80 0x5603f96ef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:34438/0 (socket says 172.21.15.32:34438) 2023-08-11T15:49:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f238959b700 1 -- 172.21.15.32:0/2098816762 learned_addr learned my addr 172.21.15.32:0/2098816762 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f2388d9a700 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9630 0x5603f96e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f2389d9c700 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f2388d9a700 1 -- 172.21.15.32:0/2098816762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96eee80 msgr2=0x5603f96ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f2388d9a700 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96eee80 0x5603f96ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f2388d9a700 1 -- 172.21.15.32:0/2098816762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 msgr2=0x5603f96f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.925+0000 7f2388d9a700 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.926+0000 7f2388d9a700 1 -- 172.21.15.32:0/2098816762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603f94bb5d0 con 0x5603f96e9630 2023-08-11T15:49:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.926+0000 7f2388d9a700 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9630 0x5603f96e9ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f23800114c0 tx=0x7f23800117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a6f75e4ad8da68f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.926+0000 7f236f7fe700 1 -- 172.21.15.32:0/2098816762 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380004230 con 0x5603f96e9630 2023-08-11T15:49:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.926+0000 7f236f7fe700 1 -- 172.21.15.32:0/2098816762 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2380004cf0 con 0x5603f96e9630 2023-08-11T15:49:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.926+0000 7f236f7fe700 1 -- 172.21.15.32:0/2098816762 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800056c0 con 0x5603f96e9630 2023-08-11T15:49:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 -- 172.21.15.32:0/2098816762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9630 msgr2=0x5603f96e9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9630 0x5603f96e9ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f23800114c0 tx=0x7f23800117d0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 -- 172.21.15.32:0/2098816762 shutdown_connections 2023-08-11T15:49:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96eee80 0x5603f96ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 --2- 172.21.15.32:0/2098816762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9630 0x5603f96e9ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 -- 172.21.15.32:0/2098816762 >> 172.21.15.32:0/2098816762 conn(0x5603f96168f0 msgr2=0x5603f96e3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 -- 172.21.15.32:0/2098816762 shutdown_connections 2023-08-11T15:49:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.927+0000 7f239f16fb80 1 -- 172.21.15.32:0/2098816762 wait complete. 2023-08-11T15:49:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.928+0000 7f239f16fb80 1 -- start start 2023-08-11T15:49:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.928+0000 7f239f16fb80 1 Processor -- start 2023-08-11T15:49:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9630 0x5603f96f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9ff0 0x5603f96f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f239f16fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f95d9a60 con 0x5603f96e9630 2023-08-11T15:49:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f239f16fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f95d9760 con 0x5603f96e9ff0 2023-08-11T15:49:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f239f16fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f94bd8c0 con 0x5603f96eee80 2023-08-11T15:49:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f239f16fb80 10 client.0 _ll_register_callbacks cb 0x5603f95dca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2388d9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9ff0 0x5603f96f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f238959b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9630 0x5603f96f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2389d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2388d9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9ff0 0x5603f96f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:34442/0 (socket says 172.21.15.32:34442) 2023-08-11T15:49:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2388d9a700 1 -- 172.21.15.32:0/127903681 learned_addr learned my addr 172.21.15.32:0/127903681 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2389d9c700 1 -- 172.21.15.32:0/127903681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9ff0 msgr2=0x5603f96f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2389d9c700 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9ff0 0x5603f96f8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2389d9c700 1 -- 172.21.15.32:0/127903681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9630 msgr2=0x5603f96f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2389d9c700 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9630 0x5603f96f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.929+0000 7f2389d9c700 1 -- 172.21.15.32:0/127903681 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2380009c90 con 0x5603f96eee80 2023-08-11T15:49:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f2389d9c700 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96f2d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f237c0c9f30 tx=0x7f237c099b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a9a8906a4f1fb7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f236cff9700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:49:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f236cff9700 1 -- 172.21.15.32:0/127903681 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c01d920 con 0x5603f96eee80 2023-08-11T15:49:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603f96dc7e0 con 0x5603f96eee80 2023-08-11T15:49:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f236cff9700 1 -- 172.21.15.32:0/127903681 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f237c099f80 con 0x5603f96eee80 2023-08-11T15:49:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f236cff9700 1 -- 172.21.15.32:0/127903681 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f237c09ddb0 con 0x5603f96eee80 2023-08-11T15:49:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f239f16fb80 10 client.25597 fetch_fsmap learned FSMap version 25 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.930+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5603f9467d80 con 0x5603f96eee80 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f236cff9700 1 -- 172.21.15.32:0/127903681 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603f96dc7e0 con 0x5603f96eee80 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f236cff9700 1 -- 172.21.15.32:0/127903681 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f237c099500 con 0x5603f96eee80 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f239f16fb80 10 client.25597 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f239f16fb80 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f239f16fb80 20 client.25597 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f239f16fb80 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9702d20 0x5603f9705110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f239f16fb80 4 client.25597 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:49:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603f96b8480 con 0x5603f9702d20 2023-08-11T15:49:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.931+0000 7f238959b700 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9702d20 0x5603f9705110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.932+0000 7f238959b700 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9702d20 0x5603f9705110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7bb0d9b14493a server_cookie=7d202a6220fe7091 in_seq=0 out_seq=0 2023-08-11T15:49:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.932+0000 7f236cff9700 10 client.25597 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.932+0000 7f236cff9700 1 -- 172.21.15.32:0/127903681 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5603f96b8480 con 0x5603f9702d20 2023-08-11T15:49:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.932+0000 7f236cff9700 10 client.25597 handle_command_reply: tid=0 2023-08-11T15:49:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.932+0000 7f239f16fb80 1 client.25597 shutdown 2023-08-11T15:49:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 msgr2=0x5603f96f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96f2d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f237c0c9f30 tx=0x7f237c099b40 comp rx=0 tx=0).stop 2023-08-11T15:49:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 shutdown_connections 2023-08-11T15:49:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9702d20 0x5603f9705110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9630 0x5603f96f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96f2d20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 --2- 172.21.15.32:0/127903681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9ff0 0x5603f96f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 >> 172.21.15.32:0/127903681 conn(0x5603f96168f0 msgr2=0x5603f96e35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f236cff9700 0 client.25597 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 shutdown_connections 2023-08-11T15:49:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 1 -- 172.21.15.32:0/127903681 wait complete. 2023-08-11T15:49:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.933+0000 7f239f16fb80 20 client.25597 trim_cache size 0 max 16384 2023-08-11T15:49:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 Processor -- start 2023-08-11T15:49:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 -- start start 2023-08-11T15:49:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f96e5db0 con 0x5603f96e9ff0 2023-08-11T15:49:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f96ecad0 con 0x5603f96e9630 2023-08-11T15:49:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f239f16fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f96f12a0 con 0x5603f96eee80 2023-08-11T15:49:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.950+0000 7f238959b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96e3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:34454/0 (socket says 172.21.15.32:34454) 2023-08-11T15:49:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 -- 172.21.15.32:0/3298282615 learned_addr learned my addr 172.21.15.32:0/3298282615 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f2388d9a700 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f2389d9c700 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 -- 172.21.15.32:0/3298282615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 msgr2=0x5603f96ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 -- 172.21.15.32:0/3298282615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 msgr2=0x5603f96eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 -- 172.21.15.32:0/3298282615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603f9450c00 con 0x5603f96e9630 2023-08-11T15:49:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f238959b700 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96e3d50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2378009870 tx=0x7f2378014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ec87ff27f6e74d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f236ffff700 1 -- 172.21.15.32:0/3298282615 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378024040 con 0x5603f96e9630 2023-08-11T15:49:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f236ffff700 1 -- 172.21.15.32:0/3298282615 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2378029470 con 0x5603f96e9630 2023-08-11T15:49:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.951+0000 7f236ffff700 1 -- 172.21.15.32:0/3298282615 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237801eb20 con 0x5603f96e9630 2023-08-11T15:49:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 -- 172.21.15.32:0/3298282615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 msgr2=0x5603f96e3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96e3d50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2378009870 tx=0x7f2378014000 comp rx=0 tx=0).stop 2023-08-11T15:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 -- 172.21.15.32:0/3298282615 shutdown_connections 2023-08-11T15:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96eee80 0x5603f96ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96e3d50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 --2- 172.21.15.32:0/3298282615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96e9ff0 0x5603f96eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 -- 172.21.15.32:0/3298282615 >> 172.21.15.32:0/3298282615 conn(0x5603f96168f0 msgr2=0x5603f96f35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 -- 172.21.15.32:0/3298282615 shutdown_connections 2023-08-11T15:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.952+0000 7f239f16fb80 1 -- 172.21.15.32:0/3298282615 wait complete. 2023-08-11T15:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.953+0000 7f239f16fb80 1 -- start start 2023-08-11T15:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.953+0000 7f239f16fb80 1 Processor -- start 2023-08-11T15:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.953+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.953+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9ff0 0x5603f96f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.953+0000 7f239f16fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96eee80 0x5603f96f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f239f16fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f96f71a0 con 0x5603f96eee80 2023-08-11T15:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f239f16fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f96f72e0 con 0x5603f96e9630 2023-08-11T15:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f239f16fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603f96f7470 con 0x5603f96e9ff0 2023-08-11T15:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f239f16fb80 10 client.0 _ll_register_callbacks cb 0x7f237c0962c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:34470/0 (socket says 172.21.15.32:34470) 2023-08-11T15:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 -- 172.21.15.32:0/1568813621 learned_addr learned my addr 172.21.15.32:0/1568813621 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f2388d9a700 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9ff0 0x5603f96f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f2389d9c700 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96eee80 0x5603f96f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 -- 172.21.15.32:0/1568813621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9ff0 msgr2=0x5603f96f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9ff0 0x5603f96f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 -- 172.21.15.32:0/1568813621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96eee80 msgr2=0x5603f96f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96eee80 0x5603f96f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 -- 172.21.15.32:0/1568813621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2378017070 con 0x5603f96e9630 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f238959b700 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96f8360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2378011ab0 tx=0x7f237802ab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=640865d5b68b5f11 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.954+0000 7f236cff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f236cff9700 1 -- 172.21.15.32:0/1568813621 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378024030 con 0x5603f96e9630 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2380004cf0 con 0x5603f96e9630 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f236cff9700 1 -- 172.21.15.32:0/1568813621 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2378005040 con 0x5603f96e9630 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f236cff9700 1 -- 172.21.15.32:0/1568813621 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f237801e4c0 con 0x5603f96e9630 2023-08-11T15:49:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 10 client.25601 fetch_fsmap learned FSMap version 25 2023-08-11T15:49:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5603f948e1f0 con 0x5603f96e9630 2023-08-11T15:49:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f236cff9700 1 -- 172.21.15.32:0/1568813621 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2380004cf0 con 0x5603f96e9630 2023-08-11T15:49:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f236cff9700 1 -- 172.21.15.32:0/1568813621 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2378004d00 con 0x5603f96e9630 2023-08-11T15:49:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 10 client.25601 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:49:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 10 client.25601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 20 client.25601 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9757f10 0x5603f975a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 4 client.25601 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:49:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.955+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603f96347c0 con 0x5603f9757f10 2023-08-11T15:49:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.956+0000 7f2388d9a700 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9757f10 0x5603f975a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.956+0000 7f2388d9a700 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9757f10 0x5603f975a330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd71800c776870f server_cookie=b97e7857d6269ce6 in_seq=0 out_seq=0 2023-08-11T15:49:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:10.956+0000 7f236cff9700 10 client.25601 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.214+0000 7f236cff9700 1 -- 172.21.15.32:0/1568813621 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603f96347c0 con 0x5603f9757f10 2023-08-11T15:49:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.214+0000 7f236cff9700 10 client.25601 handle_command_reply: tid=0 2023-08-11T15:49:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.214+0000 7f239f16fb80 1 client.25601 shutdown 2023-08-11T15:49:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 msgr2=0x5603f96f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96f8360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2378011ab0 tx=0x7f237802ab00 comp rx=0 tx=0).stop 2023-08-11T15:49:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 shutdown_connections 2023-08-11T15:49:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5603f9757f10 0x5603f975a330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5603f96eee80 0x5603f96f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f96e9630 0x5603f96f8360 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 --2- 172.21.15.32:0/1568813621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603f96e9ff0 0x5603f96f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f236cff9700 0 client.25601 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.215+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 >> 172.21.15.32:0/1568813621 conn(0x5603f96168f0 msgr2=0x5603f96edeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.217+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 shutdown_connections 2023-08-11T15:49:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.217+0000 7f239f16fb80 1 -- 172.21.15.32:0/1568813621 wait complete. 2023-08-11T15:49:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.218+0000 7f239f16fb80 20 client.25601 trim_cache size 0 max 16384 2023-08-11T15:49:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:49:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:49:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:49:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:49:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f23700a0140 msgr2=0x7f23700a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f23700a0140 0x7f23700a2600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2378009d70 tx=0x7f2378009260 comp rx=0 tx=0).stop 2023-08-11T15:49:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23841068d0 msgr2=0x7f23841a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23841068d0 0x7f23841a4a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f237c009fd0 tx=0x7f237c004c80 comp rx=0 tx=0).stop 2023-08-11T15:49:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 shutdown_connections 2023-08-11T15:49:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f23700a0140 0x7f23700a2600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384067d10 0x7f23840ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23841068d0 0x7f23841a4a80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 --2- 172.21.15.32:0/1582478074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23840686d0 0x7f2384100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 >> 172.21.15.32:0/1582478074 conn(0x7f2384075250 msgr2=0x7f23840fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 shutdown_connections 2023-08-11T15:49:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:11.219+0000 7f238b7ff700 1 -- 172.21.15.32:0/1582478074 wait complete. 2023-08-11T15:49:11.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 215746 13.56 MB/sec warmup 87 sec latency 76.282 ms 2023-08-11T15:49:12.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:12 smithi032 ceph-mon[97103]: pgmap v80: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 342 op/s 2023-08-11T15:49:12.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:12 smithi192 ceph-mon[130806]: pgmap v80: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 342 op/s 2023-08-11T15:49:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:12 smithi134 ceph-mon[130447]: pgmap v80: 129 pgs: 129 active+clean; 44 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 342 op/s 2023-08-11T15:49:12.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 218081 13.55 MB/sec warmup 88 sec latency 13.041 ms 2023-08-11T15:49:13.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 220837 13.56 MB/sec warmup 89 sec latency 6.083 ms 2023-08-11T15:49:14.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:14 smithi032 ceph-mon[97103]: pgmap v81: 129 pgs: 129 active+clean; 36 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 575 op/s 2023-08-11T15:49:14.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:14 smithi192 ceph-mon[130806]: pgmap v81: 129 pgs: 129 active+clean; 36 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 575 op/s 2023-08-11T15:49:14.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:14 smithi134 ceph-mon[130447]: pgmap v81: 129 pgs: 129 active+clean; 36 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 575 op/s 2023-08-11T15:49:14.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 223881 13.62 MB/sec warmup 90 sec latency 22.935 ms 2023-08-11T15:49:15.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 226839 13.63 MB/sec warmup 91 sec latency 6.987 ms 2023-08-11T15:49:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:16 smithi032 ceph-mon[97103]: pgmap v82: 129 pgs: 129 active+clean; 36 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 317 op/s 2023-08-11T15:49:16.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:16 smithi192 ceph-mon[130806]: pgmap v82: 129 pgs: 129 active+clean; 36 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 317 op/s 2023-08-11T15:49:16.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:16 smithi134 ceph-mon[130447]: pgmap v82: 129 pgs: 129 active+clean; 36 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 317 op/s 2023-08-11T15:49:16.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 228036 13.54 MB/sec warmup 92 sec latency 185.734 ms 2023-08-11T15:49:17.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 229445 13.46 MB/sec warmup 93 sec latency 380.825 ms 2023-08-11T15:49:18.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:17 smithi134 ceph-mon[130447]: pgmap v83: 129 pgs: 129 active+clean; 48 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 401 op/s 2023-08-11T15:49:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:17 smithi032 ceph-mon[97103]: pgmap v83: 129 pgs: 129 active+clean; 48 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 401 op/s 2023-08-11T15:49:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:17 smithi192 ceph-mon[130806]: pgmap v83: 129 pgs: 129 active+clean; 48 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 401 op/s 2023-08-11T15:49:18.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 231224 13.45 MB/sec warmup 94 sec latency 217.802 ms 2023-08-11T15:49:19.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234160 13.49 MB/sec warmup 95 sec latency 23.383 ms 2023-08-11T15:49:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:20 smithi032 ceph-mon[97103]: pgmap v84: 129 pgs: 129 active+clean; 55 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2023-08-11T15:49:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:20 smithi192 ceph-mon[130806]: pgmap v84: 129 pgs: 129 active+clean; 55 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2023-08-11T15:49:20.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:20 smithi134 ceph-mon[130447]: pgmap v84: 129 pgs: 129 active+clean; 55 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2023-08-11T15:49:20.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 236555 13.44 MB/sec warmup 96 sec latency 19.673 ms 2023-08-11T15:49:21.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 238519 13.43 MB/sec warmup 97 sec latency 69.924 ms 2023-08-11T15:49:22.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:22 smithi032 ceph-mon[97103]: pgmap v85: 129 pgs: 129 active+clean; 55 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2023-08-11T15:49:22.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:22 smithi192 ceph-mon[130806]: pgmap v85: 129 pgs: 129 active+clean; 55 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2023-08-11T15:49:22.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:22 smithi134 ceph-mon[130447]: pgmap v85: 129 pgs: 129 active+clean; 55 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2023-08-11T15:49:22.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 241064 13.43 MB/sec warmup 98 sec latency 102.039 ms 2023-08-11T15:49:23.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 243134 13.40 MB/sec warmup 99 sec latency 30.400 ms 2023-08-11T15:49:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 -- 172.21.15.32:0/3582625566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e41068d0 msgr2=0x7fc0e4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/3582625566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e41068d0 0x7fc0e4106cb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009990 tx=0x7fc0d4009ca0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 -- 172.21.15.32:0/3582625566 shutdown_connections 2023-08-11T15:49:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/3582625566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e40686d0 0x7fc0e410d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/3582625566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e41068d0 0x7fc0e4106cb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/3582625566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e4067d10 0x7fc0e4068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 -- 172.21.15.32:0/3582625566 >> 172.21.15.32:0/3582625566 conn(0x7fc0e4075250 msgr2=0x7fc0e4075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 -- 172.21.15.32:0/3582625566 shutdown_connections 2023-08-11T15:49:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.375+0000 7fc0e9dff700 1 -- 172.21.15.32:0/3582625566 wait complete. 2023-08-11T15:49:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.376+0000 7fc0e9dff700 1 Processor -- start 2023-08-11T15:49:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.376+0000 7fc0e9dff700 1 -- start start 2023-08-11T15:49:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.376+0000 7fc0e9dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e4067d10 0x7fc0e419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e9dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e40686d0 0x7fc0e419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e9dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e41068d0 0x7fc0e4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e9dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e41992e0 con 0x7fc0e40686d0 2023-08-11T15:49:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e9dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e4199450 con 0x7fc0e41068d0 2023-08-11T15:49:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e9dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e4199610 con 0x7fc0e4067d10 2023-08-11T15:49:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e41068d0 0x7fc0e4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e40686d0 0x7fc0e419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e41068d0 0x7fc0e4198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:42952/0 (socket says 172.21.15.32:42952) 2023-08-11T15:49:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 -- 172.21.15.32:0/25572504 learned_addr learned my addr 172.21.15.32:0/25572504 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e4067d10 0x7fc0e419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 -- 172.21.15.32:0/25572504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e4067d10 msgr2=0x7fc0e419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e4067d10 0x7fc0e419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 -- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e40686d0 msgr2=0x7fc0e419f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e40686d0 0x7fc0e419f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.377+0000 7fc0e3fff700 1 -- 172.21.15.32:0/25572504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d40095f0 con 0x7fc0e41068d0 2023-08-11T15:49:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.378+0000 7fc0e3fff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e41068d0 0x7fc0e4198b10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc007ee0 tx=0x7fc0dc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.378+0000 7fc0e0ff9700 1 -- 172.21.15.32:0/25572504 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00dd20 con 0x7fc0e41068d0 2023-08-11T15:49:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.378+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e41998f0 con 0x7fc0e41068d0 2023-08-11T15:49:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.378+0000 7fc0e0ff9700 1 -- 172.21.15.32:0/25572504 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0dc00f040 con 0x7fc0e41068d0 2023-08-11T15:49:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.378+0000 7fc0e0ff9700 1 -- 172.21.15.32:0/25572504 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0048a0 con 0x7fc0e41068d0 2023-08-11T15:49:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.378+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e41a5cc0 con 0x7fc0e41068d0 2023-08-11T15:49:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.381+0000 7fc0e0ff9700 1 -- 172.21.15.32:0/25572504 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fc0dc003a00 con 0x7fc0e41068d0 2023-08-11T15:49:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 Processor -- start 2023-08-11T15:49:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 -- start start 2023-08-11T15:49:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4aae80 0x5568ec4ab260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec3fe5d0 con 0x5568ec4a5630 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec3fbef0 con 0x5568ec4aae80 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.383+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec3f77e0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e0ff9700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc0cc0a00c0 0x7fc0cc0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4aae80 0x5568ec4ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e0ff9700 1 -- 172.21.15.32:0/25572504 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc0dc0cb190 con 0x7fc0e41068d0 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4aae80 0x5568ec4ab260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:42956/0 (socket says 172.21.15.32:42956) 2023-08-11T15:49:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e2ffd700 1 -- 172.21.15.32:0/371079203 learned_addr learned my addr 172.21.15.32:0/371079203 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e3fff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc0cc0a00c0 0x7fc0cc0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e37fe700 1 -- 172.21.15.32:0/371079203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 msgr2=0x5568ec4b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e37fe700 1 -- 172.21.15.32:0/371079203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4aae80 msgr2=0x5568ec4ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4aae80 0x5568ec4ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e37fe700 1 -- 172.21.15.32:0/371079203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568ec2775d0 con 0x5568ec4a5630 2023-08-11T15:49:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.384+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4aae80 0x5568ec4ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:49:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.385+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4a5ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4000c00 tx=0x7fc0d400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639b8d8be466fb3d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.385+0000 7fc0e3fff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc0cc0a00c0 0x7fc0cc0a2580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc09fe10 tx=0x7fc0dc00df10 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.385+0000 7fc0c9ffb700 1 -- 172.21.15.32:0/371079203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d401d070 con 0x5568ec4a5630 2023-08-11T15:49:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.385+0000 7fc0c9ffb700 1 -- 172.21.15.32:0/371079203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d401e040 con 0x5568ec4a5630 2023-08-11T15:49:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.385+0000 7fc0c9ffb700 1 -- 172.21.15.32:0/371079203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4004760 con 0x5568ec4a5630 2023-08-11T15:49:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/371079203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 msgr2=0x5568ec4a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4a5ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4000c00 tx=0x7fc0d400b610 comp rx=0 tx=0).stop 2023-08-11T15:49:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/371079203 shutdown_connections 2023-08-11T15:49:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4aae80 0x5568ec4ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/371079203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4a5ab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/371079203 >> 172.21.15.32:0/371079203 conn(0x5568ec3d28f0 msgr2=0x5568ec49fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/371079203 shutdown_connections 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.386+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/371079203 wait complete. 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.387+0000 7fc0fd7cab80 1 -- start start 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.387+0000 7fc0fd7cab80 1 Processor -- start 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec4aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec395a60 con 0x5568ec4a5630 2023-08-11T15:49:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec395760 con 0x5568ec4a5ff0 2023-08-11T15:49:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec2798c0 con 0x5568ec4aae80 2023-08-11T15:49:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0fd7cab80 10 client.0 _ll_register_callbacks cb 0x5568ec398a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59704/0 (socket says 172.21.15.32:59704) 2023-08-11T15:49:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e2ffd700 1 -- 172.21.15.32:0/497100589 learned_addr learned my addr 172.21.15.32:0/497100589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec4aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e37fe700 1 -- 172.21.15.32:0/497100589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 msgr2=0x5568ec4aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec4aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e37fe700 1 -- 172.21.15.32:0/497100589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 msgr2=0x5568ec4b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e37fe700 1 -- 172.21.15.32:0/497100589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d8019040 con 0x5568ec4a5ff0 2023-08-11T15:49:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.388+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec4aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:49:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b4e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4005f50 tx=0x7fc0d400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3734df7103a02597 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:49:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/497100589 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d401d040 con 0x5568ec4a5ff0 2023-08-11T15:49:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568ec4987e0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/497100589 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc0d4005980 con 0x5568ec4a5ff0 2023-08-11T15:49:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/497100589 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0d4005460 con 0x5568ec4a5ff0 2023-08-11T15:49:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0fd7cab80 10 client.25613 fetch_fsmap learned FSMap version 25 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.389+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fc0e410e020 con 0x5568ec4a5ff0 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/497100589 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568ec4987e0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/497100589 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0d4004ab0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0fd7cab80 10 client.25613 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0fd7cab80 10 client.25613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0fd7cab80 20 client.25613 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec4be990 0x5568ec4c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0fd7cab80 4 client.25613 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:49:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568ec474480 con 0x5568ec4be990 2023-08-11T15:49:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.390+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec4be990 0x5568ec4c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.391+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec4be990 0x5568ec4c0db0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462e31046c70c763 server_cookie=b6b21efd755b765e in_seq=0 out_seq=0 2023-08-11T15:49:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.391+0000 7fc0c2ffd700 10 client.25613 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.391+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/497100589 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5568ec474480 con 0x5568ec4be990 2023-08-11T15:49:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.391+0000 7fc0c2ffd700 10 client.25613 handle_command_reply: tid=0 2023-08-11T15:49:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.391+0000 7fc0fd7cab80 1 client.25613 shutdown 2023-08-11T15:49:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5ff0 msgr2=0x5568ec4b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b4e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4005f50 tx=0x7fc0d400f690 comp rx=0 tx=0).stop 2023-08-11T15:49:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 shutdown_connections 2023-08-11T15:49:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec4be990 0x5568ec4c0db0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5630 0x5568ec4b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec4aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/497100589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b4e60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0c2ffd700 0 client.25613 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 >> 172.21.15.32:0/497100589 conn(0x5568ec3d28f0 msgr2=0x5568ec49f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 shutdown_connections 2023-08-11T15:49:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/497100589 wait complete. 2023-08-11T15:49:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.392+0000 7fc0fd7cab80 20 client.25613 trim_cache size 0 max 16384 2023-08-11T15:49:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 Processor -- start 2023-08-11T15:49:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 -- start start 2023-08-11T15:49:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5ff0 0x5568ec4aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec49fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec4a8870 con 0x5568ec4a5ff0 2023-08-11T15:49:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec4a1930 con 0x5568ec4a5630 2023-08-11T15:49:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.410+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec498570 con 0x5568ec4aae80 2023-08-11T15:49:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:42978/0 (socket says 172.21.15.32:42978) 2023-08-11T15:49:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e2ffd700 1 -- 172.21.15.32:0/4110794580 learned_addr learned my addr 172.21.15.32:0/4110794580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5ff0 0x5568ec4aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec49fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e37fe700 1 -- 172.21.15.32:0/4110794580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 msgr2=0x5568ec49fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec49fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e37fe700 1 -- 172.21.15.32:0/4110794580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 msgr2=0x5568ec4c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e37fe700 1 -- 172.21.15.32:0/4110794580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568ec223d80 con 0x5568ec4a5ff0 2023-08-11T15:49:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:49:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.411+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5ff0 0x5568ec4aec00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40093f0 tx=0x7fc0d4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c506e298f9f525d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0ca7fc700 1 -- 172.21.15.32:0/4110794580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40226a0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0ca7fc700 1 -- 172.21.15.32:0/4110794580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d40059e0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0ca7fc700 1 -- 172.21.15.32:0/4110794580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d402b7d0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/4110794580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5ff0 msgr2=0x5568ec4aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5ff0 0x5568ec4aec00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40093f0 tx=0x7fc0d4009ca0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/4110794580 shutdown_connections 2023-08-11T15:49:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4aae80 0x5568ec49fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/4110794580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4a5ff0 0x5568ec4aec00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/4110794580 >> 172.21.15.32:0/4110794580 conn(0x5568ec3d28f0 msgr2=0x5568ec4ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.412+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/4110794580 shutdown_connections 2023-08-11T15:49:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.413+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/4110794580 wait complete. 2023-08-11T15:49:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.413+0000 7fc0fd7cab80 1 -- start start 2023-08-11T15:49:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.413+0000 7fc0fd7cab80 1 Processor -- start 2023-08-11T15:49:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0fd7cab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4aae80 0x5568ec511600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec4b3290 con 0x5568ec4aae80 2023-08-11T15:49:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec511b40 con 0x5568ec4a5630 2023-08-11T15:49:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0fd7cab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ec511cb0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0fd7cab80 10 client.0 _ll_register_callbacks cb 0x7fc0d8009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:42986/0 (socket says 172.21.15.32:42986) 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:35816/0 (socket says 172.21.15.32:35816) 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e2ffd700 1 -- 172.21.15.32:0/1116738285 learned_addr learned my addr 172.21.15.32:0/1116738285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4aae80 0x5568ec511600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e37fe700 1 -- 172.21.15.32:0/1116738285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 msgr2=0x5568ec4b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e37fe700 1 -- 172.21.15.32:0/1116738285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4aae80 msgr2=0x5568ec511600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4aae80 0x5568ec511600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e37fe700 1 -- 172.21.15.32:0/1116738285 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d80239e0 con 0x5568ec4a5ff0 2023-08-11T15:49:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.414+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4aae80 0x5568ec511600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0e37fe700 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b2b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x5568ec4b0390 tx=0x7fc0d4022740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35fd7fa5d39992aa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/1116738285 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d401d070 con 0x5568ec4a5ff0 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0d401e040 con 0x5568ec4a5ff0 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/1116738285 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fc0d4000e40 con 0x5568ec4a5ff0 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/1116738285 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0d4005630 con 0x5568ec4a5ff0 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0fd7cab80 10 client.25621 fetch_fsmap learned FSMap version 25 2023-08-11T15:49:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fc0e410e020 con 0x5568ec4a5ff0 2023-08-11T15:49:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/1116738285 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d401e040 con 0x5568ec4a5ff0 2023-08-11T15:49:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.415+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/1116738285 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0d4004500 con 0x5568ec4a5ff0 2023-08-11T15:49:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0fd7cab80 10 client.25621 fetch_fsmap finished waiting for FSMap version 25 2023-08-11T15:49:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0fd7cab80 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0fd7cab80 20 client.25621 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec512f60 0x5568ec515350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0fd7cab80 4 client.25621 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:49:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5568ec3f07c0 con 0x5568ec512f60 2023-08-11T15:49:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec512f60 0x5568ec515350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0e2ffd700 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec512f60 0x5568ec515350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe393038c038f262 server_cookie=d13ce7d2263edfd1 in_seq=0 out_seq=0 2023-08-11T15:49:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.416+0000 7fc0c2ffd700 10 client.25621 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:24.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:24 smithi032 ceph-mon[97103]: pgmap v86: 129 pgs: 129 active+clean; 48 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 594 op/s 2023-08-11T15:49:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:24 smithi192 ceph-mon[130806]: pgmap v86: 129 pgs: 129 active+clean; 48 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 594 op/s 2023-08-11T15:49:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.595+0000 7fc0c2ffd700 1 -- 172.21.15.32:0/1116738285 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5568ec3f07c0 con 0x5568ec512f60 2023-08-11T15:49:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.595+0000 7fc0c2ffd700 10 client.25621 handle_command_reply: tid=0 2023-08-11T15:49:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.595+0000 7fc0fd7cab80 1 client.25621 shutdown 2023-08-11T15:49:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.595+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 msgr2=0x5568ec4b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.595+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b2b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x5568ec4b0390 tx=0x7fc0d4022740 comp rx=0 tx=0).stop 2023-08-11T15:49:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.595+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 shutdown_connections 2023-08-11T15:49:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5568ec512f60 0x5568ec515350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568ec4aae80 0x5568ec511600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568ec4a5630 0x5568ec4b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0c2ffd700 0 client.25621 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 1 --2- 172.21.15.32:0/1116738285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568ec4a5ff0 0x5568ec4b2b00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 >> 172.21.15.32:0/1116738285 conn(0x5568ec3d28f0 msgr2=0x5568ec4a9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 shutdown_connections 2023-08-11T15:49:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 1 -- 172.21.15.32:0/1116738285 wait complete. 2023-08-11T15:49:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.596+0000 7fc0fd7cab80 20 client.25621 trim_cache size 0 max 16384 2023-08-11T15:49:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:49:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:49:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:49:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:49:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc0cc0a00c0 msgr2=0x7fc0cc0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc0cc0a00c0 0x7fc0cc0a2580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc09fe10 tx=0x7fc0dc00df10 comp rx=0 tx=0).stop 2023-08-11T15:49:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e41068d0 msgr2=0x7fc0e4198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e41068d0 0x7fc0e4198b10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc007ee0 tx=0x7fc0dc00d3b0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 shutdown_connections 2023-08-11T15:49:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc0cc0a00c0 0x7fc0cc0a2580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0e41068d0 0x7fc0e4198b10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e4067d10 0x7fc0e419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 --2- 172.21.15.32:0/25572504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0e40686d0 0x7fc0e419f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 >> 172.21.15.32:0/25572504 conn(0x7fc0e4075250 msgr2=0x7fc0e410baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 shutdown_connections 2023-08-11T15:49:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:24.597+0000 7fc0e9dff700 1 -- 172.21.15.32:0/25572504 wait complete. 2023-08-11T15:49:24.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:24 smithi134 ceph-mon[130447]: pgmap v86: 129 pgs: 129 active+clean; 48 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 594 op/s 2023-08-11T15:49:24.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 245387 13.40 MB/sec warmup 100 sec latency 102.139 ms 2023-08-11T15:49:25.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:49:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setuser ceph since I am not root 2023-08-11T15:49:25.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:49:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setgroup ceph since I am not root 2023-08-11T15:49:25.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:49:24 smithi032 ceph-mds[131163]: -- 172.21.15.32:0/785401792 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621485071e0 con 0x5621484dc800 2023-08-11T15:49:25.054 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:49:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: starting mds.a at 2023-08-11T15:49:25.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:25 smithi032 ceph-mon[97103]: daemon mds.a restarted 2023-08-11T15:49:25.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:25 smithi192 ceph-mon[130806]: daemon mds.a restarted 2023-08-11T15:49:25.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:25 smithi134 ceph-mon[130447]: daemon mds.a restarted 2023-08-11T15:49:25.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 248463 13.45 MB/sec warmup 101 sec latency 14.515 ms 2023-08-11T15:49:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:26 smithi032 ceph-mon[97103]: pgmap v87: 129 pgs: 129 active+clean; 48 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2023-08-11T15:49:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:26 smithi032 ceph-mon[97103]: osdmap e96: 12 total, 12 up, 12 in 2023-08-11T15:49:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:26 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6840/1734048420,v1:172.21.15.32:6841/1734048420] up:boot 2023-08-11T15:49:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:26 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:49:26.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:26 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:49:26.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:26 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:49:26.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:26 smithi192 ceph-mon[130806]: pgmap v87: 129 pgs: 129 active+clean; 48 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2023-08-11T15:49:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:26 smithi192 ceph-mon[130806]: osdmap e96: 12 total, 12 up, 12 in 2023-08-11T15:49:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:26 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6840/1734048420,v1:172.21.15.32:6841/1734048420] up:boot 2023-08-11T15:49:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:26 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:49:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:26 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:49:26.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:26 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:49:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:26 smithi134 ceph-mon[130447]: pgmap v87: 129 pgs: 129 active+clean; 48 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2023-08-11T15:49:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:26 smithi134 ceph-mon[130447]: osdmap e96: 12 total, 12 up, 12 in 2023-08-11T15:49:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:26 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6840/1734048420,v1:172.21.15.32:6841/1734048420] up:boot 2023-08-11T15:49:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:26 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:49:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:26 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:49:26.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:26 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:49:26.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 250664 13.39 MB/sec warmup 102 sec latency 35.569 ms 2023-08-11T15:49:27.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 252670 13.39 MB/sec warmup 103 sec latency 81.647 ms 2023-08-11T15:49:28.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:27 smithi134 ceph-mon[130447]: pgmap v89: 129 pgs: 129 active+clean; 36 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 558 op/s 2023-08-11T15:49:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:27 smithi032 ceph-mon[97103]: pgmap v89: 129 pgs: 129 active+clean; 36 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 558 op/s 2023-08-11T15:49:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:27 smithi192 ceph-mon[130806]: pgmap v89: 129 pgs: 129 active+clean; 36 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 558 op/s 2023-08-11T15:49:28.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 255742 13.44 MB/sec warmup 104 sec latency 19.204 ms 2023-08-11T15:49:29.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 258628 13.44 MB/sec warmup 105 sec latency 6.973 ms 2023-08-11T15:49:30.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:30 smithi032 ceph-mon[97103]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 578 op/s 2023-08-11T15:49:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:30 smithi192 ceph-mon[130806]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 578 op/s 2023-08-11T15:49:30.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:30 smithi134 ceph-mon[130447]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 578 op/s 2023-08-11T15:49:30.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 261173 13.42 MB/sec warmup 106 sec latency 23.790 ms 2023-08-11T15:49:31.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 263456 13.43 MB/sec warmup 107 sec latency 36.458 ms 2023-08-11T15:49:32.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:32 smithi032 ceph-mon[97103]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 578 op/s 2023-08-11T15:49:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:32 smithi192 ceph-mon[130806]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 578 op/s 2023-08-11T15:49:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:32 smithi134 ceph-mon[130447]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.2 MiB/s wr, 578 op/s 2023-08-11T15:49:32.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 266336 13.47 MB/sec warmup 108 sec latency 55.299 ms 2023-08-11T15:49:33.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 268662 13.42 MB/sec warmup 109 sec latency 85.626 ms 2023-08-11T15:49:34.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:34 smithi032 ceph-mon[97103]: pgmap v92: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2023-08-11T15:49:34.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:34 smithi192 ceph-mon[130806]: pgmap v92: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2023-08-11T15:49:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:34 smithi134 ceph-mon[130447]: pgmap v92: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2023-08-11T15:49:34.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 271095 13.45 MB/sec warmup 110 sec latency 6.598 ms 2023-08-11T15:49:35.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 273753 13.45 MB/sec warmup 111 sec latency 19.013 ms 2023-08-11T15:49:36.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:36 smithi032 ceph-mon[97103]: pgmap v93: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2023-08-11T15:49:36.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:36 smithi192 ceph-mon[130806]: pgmap v93: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2023-08-11T15:49:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:36 smithi134 ceph-mon[130447]: pgmap v93: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2023-08-11T15:49:36.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 276854 13.49 MB/sec warmup 112 sec latency 7.429 ms 2023-08-11T15:49:37.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 279467 13.48 MB/sec warmup 113 sec latency 7.042 ms 2023-08-11T15:49:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/1519718163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 msgr2=0x7f5e48106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/1519718163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 0x7f5e48106c90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009990 tx=0x7f5e44009ca0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/1519718163 shutdown_connections 2023-08-11T15:49:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/1519718163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e48101260 0x7f5e4810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/1519718163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 0x7f5e48106c90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/1519718163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e481008a0 0x7f5e48100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/1519718163 >> 172.21.15.32:0/1519718163 conn(0x7f5e480fc370 msgr2=0x7f5e480fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/1519718163 shutdown_connections 2023-08-11T15:49:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.765+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/1519718163 wait complete. 2023-08-11T15:49:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.766+0000 7f5e4f7ff700 1 Processor -- start 2023-08-11T15:49:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.766+0000 7f5e4f7ff700 1 -- start start 2023-08-11T15:49:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4f7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e481008a0 0x7f5e481a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4f7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e48101260 0x7f5e481a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4f7ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 0x7f5e4810a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4f7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4810a9c0 con 0x7f5e481008a0 2023-08-11T15:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4f7ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4810ab30 con 0x7f5e481068b0 2023-08-11T15:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4f7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4810acf0 con 0x7f5e48101260 2023-08-11T15:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e481008a0 0x7f5e481a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e481008a0 0x7f5e481a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45906/0 (socket says 172.21.15.32:45906) 2023-08-11T15:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 -- 172.21.15.32:0/2188144773 learned_addr learned my addr 172.21.15.32:0/2188144773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e48101260 0x7f5e481a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4dd9c700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 0x7f5e4810a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 -- 172.21.15.32:0/2188144773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e48101260 msgr2=0x7f5e481a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e48101260 0x7f5e481a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 -- 172.21.15.32:0/2188144773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 msgr2=0x7f5e4810a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 0x7f5e4810a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.767+0000 7f5e4d59b700 1 -- 172.21.15.32:0/2188144773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440095f0 con 0x7f5e481008a0 2023-08-11T15:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.768+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e481008a0 0x7f5e481a1e40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005910 tx=0x7f5e44004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.768+0000 7f5e3e7fc700 1 -- 172.21.15.32:0/2188144773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4401d070 con 0x7f5e481008a0 2023-08-11T15:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.768+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4810af10 con 0x7f5e481008a0 2023-08-11T15:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.768+0000 7f5e3e7fc700 1 -- 172.21.15.32:0/2188144773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e44022470 con 0x7f5e481008a0 2023-08-11T15:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.768+0000 7f5e3e7fc700 1 -- 172.21.15.32:0/2188144773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44017650 con 0x7f5e481008a0 2023-08-11T15:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.768+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e481a9f40 con 0x7f5e481008a0 2023-08-11T15:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.772+0000 7f5e3e7fc700 1 -- 172.21.15.32:0/2188144773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f5e440225e0 con 0x7f5e481008a0 2023-08-11T15:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.773+0000 7f5e3e7fc700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f5e340a00c0 0x7f5e340a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.773+0000 7f5e630a7b80 1 Processor -- start 2023-08-11T15:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e3e7fc700 1 -- 172.21.15.32:0/2188144773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5e440ccb60 con 0x7f5e481008a0 2023-08-11T15:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f5e340a00c0 0x7f5e340a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e630a7b80 1 -- start start 2023-08-11T15:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a93e80 0x55e0f0a94260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a8eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f09e75d0 con 0x55e0f0a8eff0 2023-08-11T15:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f09e4ef0 con 0x55e0f0a93e80 2023-08-11T15:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.774+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f09e07e0 con 0x55e0f0a8e630 2023-08-11T15:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f5e340a00c0 0x7f5e340a2580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e4819e860 tx=0x7f5e38009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a93e80 0x55e0f0a94260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a93e80 0x55e0f0a94260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:46994/0 (socket says 172.21.15.32:46994) 2023-08-11T15:49:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3149287861 learned_addr learned my addr 172.21.15.32:0/3149287861 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a8eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4dd9c700 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3149287861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 msgr2=0x55e0f0a8eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a8eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3149287861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 msgr2=0x55e0f0a9ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3149287861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f08605d0 con 0x55e0f0a93e80 2023-08-11T15:49:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4dd9c700 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:49:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a93e80 0x55e0f0a94260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e44099f20 tx=0x7f5e44099b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ab04cfa8e95d88 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e337fe700 1 -- 172.21.15.32:0/3149287861 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44099f90 con 0x55e0f0a93e80 2023-08-11T15:49:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e337fe700 1 -- 172.21.15.32:0/3149287861 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4409ddc0 con 0x55e0f0a93e80 2023-08-11T15:49:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.775+0000 7f5e337fe700 1 -- 172.21.15.32:0/3149287861 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44099480 con 0x55e0f0a93e80 2023-08-11T15:49:37.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3149287861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a93e80 msgr2=0x55e0f0a94260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a93e80 0x55e0f0a94260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e44099f20 tx=0x7f5e44099b50 comp rx=0 tx=0).stop 2023-08-11T15:49:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3149287861 shutdown_connections 2023-08-11T15:49:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a93e80 0x55e0f0a94260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3149287861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a8eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3149287861 >> 172.21.15.32:0/3149287861 conn(0x55e0f09bb8f0 msgr2=0x55e0f0a88e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3149287861 shutdown_connections 2023-08-11T15:49:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.776+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3149287861 wait complete. 2023-08-11T15:49:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.777+0000 7f5e630a7b80 1 -- start start 2023-08-11T15:49:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.777+0000 7f5e630a7b80 1 Processor -- start 2023-08-11T15:49:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f097ea60 con 0x55e0f0a8eff0 2023-08-11T15:49:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f097e760 con 0x55e0f0a8e630 2023-08-11T15:49:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f08628c0 con 0x55e0f0a93e80 2023-08-11T15:49:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e630a7b80 10 client.0 _ll_register_callbacks cb 0x55e0f0981a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:47006/0 (socket says 172.21.15.32:47006) 2023-08-11T15:49:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.778+0000 7f5e4d59b700 1 -- 172.21.15.32:0/681477120 learned_addr learned my addr 172.21.15.32:0/681477120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4dd9c700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4d59b700 1 -- 172.21.15.32:0/681477120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a93e80 msgr2=0x55e0f0a97960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4d59b700 1 -- 172.21.15.32:0/681477120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 msgr2=0x55e0f0a9db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4d59b700 1 -- 172.21.15.32:0/681477120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e481a9f40 con 0x55e0f0a8e630 2023-08-11T15:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5e440a17e0 tx=0x7f5e4409a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c9b9cf3a6862ba server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e30ff9700 1 -- 172.21.15.32:0/681477120 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4409a400 con 0x55e0f0a8e630 2023-08-11T15:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0f0a817e0 con 0x55e0f0a8e630 2023-08-11T15:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e30ff9700 1 -- 172.21.15.32:0/681477120 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5e440a1880 con 0x55e0f0a8e630 2023-08-11T15:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e30ff9700 1 -- 172.21.15.32:0/681477120 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e440a1a40 con 0x55e0f0a8e630 2023-08-11T15:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e630a7b80 10 client.25637 fetch_fsmap learned FSMap version 27 2023-08-11T15:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.779+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0f080cd80 con 0x55e0f0a8e630 2023-08-11T15:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e30ff9700 1 -- 172.21.15.32:0/681477120 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0f0a817e0 con 0x55e0f0a8e630 2023-08-11T15:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e30ff9700 1 -- 172.21.15.32:0/681477120 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5e44095d80 con 0x55e0f0a8e630 2023-08-11T15:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e630a7b80 10 client.25637 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e630a7b80 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e630a7b80 20 client.25637 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0aa75b0 0x55e0f0aa99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e630a7b80 4 client.25637 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0f0a5d480 con 0x55e0f0aa75b0 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.780+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0aa75b0 0x55e0f0aa99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.781+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0aa75b0 0x55e0f0aa99a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1851f2bef6e842cf server_cookie=812fe6bd5aedb9b0 in_seq=0 out_seq=0 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.781+0000 7f5e30ff9700 10 client.25637 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e30ff9700 1 -- 172.21.15.32:0/681477120 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e0f0a5d480 con 0x55e0f0aa75b0 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e30ff9700 10 client.25637 handle_command_reply: tid=0 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 client.25637 shutdown 2023-08-11T15:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 msgr2=0x55e0f0a9d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5e440a17e0 tx=0x7f5e4409a750 comp rx=0 tx=0).stop 2023-08-11T15:49:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 shutdown_connections 2023-08-11T15:49:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0aa75b0 0x55e0f0aa99a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d630 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/681477120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e30ff9700 0 client.25637 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.782+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 >> 172.21.15.32:0/681477120 conn(0x55e0f09bb8f0 msgr2=0x55e0f0a885d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.783+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 shutdown_connections 2023-08-11T15:49:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.783+0000 7f5e630a7b80 1 -- 172.21.15.32:0/681477120 wait complete. 2023-08-11T15:49:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.783+0000 7f5e630a7b80 20 client.25637 trim_cache size 0 max 16384 2023-08-11T15:49:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.799+0000 7f5e630a7b80 1 Processor -- start 2023-08-11T15:49:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e630a7b80 1 -- start start 2023-08-11T15:49:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a88d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f0a89760 con 0x55e0f0a93e80 2023-08-11T15:49:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f0a817c0 con 0x55e0f0a8eff0 2023-08-11T15:49:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f0a89a60 con 0x55e0f0a8e630 2023-08-11T15:49:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e4dd9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e4cd9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a88d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e4dd9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45926/0 (socket says 172.21.15.32:45926) 2023-08-11T15:49:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.800+0000 7f5e4dd9c700 1 -- 172.21.15.32:0/3204181958 learned_addr learned my addr 172.21.15.32:0/3204181958 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e4cd9a700 1 -- 172.21.15.32:0/3204181958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 msgr2=0x55e0f0a88d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3204181958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a88d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e4cd9a700 1 -- 172.21.15.32:0/3204181958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 msgr2=0x55e0f0a97200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3204181958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e4cd9a700 1 -- 172.21.15.32:0/3204181958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f07f5c00 con 0x55e0f0a8eff0 2023-08-11T15:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3204181958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a95270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55e0f0aaa280 tx=0x7f5e38014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708cf31ba9271424 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e33fff700 1 -- 172.21.15.32:0/3204181958 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38022040 con 0x55e0f0a8eff0 2023-08-11T15:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e33fff700 1 -- 172.21.15.32:0/3204181958 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e38011b80 con 0x55e0f0a8eff0 2023-08-11T15:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.801+0000 7f5e33fff700 1 -- 172.21.15.32:0/3204181958 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800e6e0 con 0x55e0f0a8eff0 2023-08-11T15:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3204181958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8eff0 msgr2=0x55e0f0a95270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3204181958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a95270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55e0f0aaa280 tx=0x7f5e38014000 comp rx=0 tx=0).stop 2023-08-11T15:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3204181958 shutdown_connections 2023-08-11T15:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3204181958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a97200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3204181958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8e630 0x55e0f0a88d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3204181958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a95270 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3204181958 >> 172.21.15.32:0/3204181958 conn(0x55e0f09bb8f0 msgr2=0x55e0f0a923d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3204181958 shutdown_connections 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.802+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3204181958 wait complete. 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.803+0000 7f5e630a7b80 1 -- start start 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.803+0000 7f5e630a7b80 1 Processor -- start 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.803+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.803+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e630a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a9ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f0a9c160 con 0x55e0f0a93e80 2023-08-11T15:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f0a9c2d0 con 0x55e0f0a8e630 2023-08-11T15:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e630a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f0afb240 con 0x55e0f0a8eff0 2023-08-11T15:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e630a7b80 10 client.0 _ll_register_callbacks cb 0x7f5e44099df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:47026/0 (socket says 172.21.15.32:47026) 2023-08-11T15:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4dd9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a9ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3032295437 learned_addr learned my addr 172.21.15.32:0/3032295437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3032295437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8eff0 msgr2=0x55e0f0a9b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3032295437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 msgr2=0x55e0f0a9ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a9ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 -- 172.21.15.32:0/3032295437 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e38017070 con 0x55e0f0a8e630 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e4d59b700 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e44099b90 tx=0x7f5e440ca020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58130d82b56d51cf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.804+0000 7f5e30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:49:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e30ff9700 1 -- 172.21.15.32:0/3032295437 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44094da0 con 0x55e0f0a8e630 2023-08-11T15:49:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e30ff9700 1 -- 172.21.15.32:0/3032295437 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f5e4409a400 con 0x55e0f0a8e630 2023-08-11T15:49:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e4409ddc0 con 0x55e0f0a8e630 2023-08-11T15:49:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e30ff9700 1 -- 172.21.15.32:0/3032295437 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e440a1d10 con 0x55e0f0a8e630 2023-08-11T15:49:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e630a7b80 10 client.25643 fetch_fsmap learned FSMap version 27 2023-08-11T15:49:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0f08331f0 con 0x55e0f0a8e630 2023-08-11T15:49:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e30ff9700 1 -- 172.21.15.32:0/3032295437 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4409ddc0 con 0x55e0f0a8e630 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e30ff9700 1 -- 172.21.15.32:0/3032295437 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5e44098ef0 con 0x55e0f0a8e630 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e630a7b80 10 client.25643 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e630a7b80 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e630a7b80 20 client.25643 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.805+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0afc3c0 0x55e0f0afe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.806+0000 7f5e630a7b80 4 client.25643 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.806+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0f09d97c0 con 0x55e0f0afc3c0 2023-08-11T15:49:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.806+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0afc3c0 0x55e0f0afe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.806+0000 7f5e30ff9700 10 client.25643 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:37.806+0000 7f5e4cd9a700 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0afc3c0 0x55e0f0afe7b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4f74418376e523 server_cookie=e9f2e95c6da58c59 in_seq=0 out_seq=0 2023-08-11T15:49:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e30ff9700 1 -- 172.21.15.32:0/3032295437 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0f09d97c0 con 0x55e0f0afc3c0 2023-08-11T15:49:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e30ff9700 10 client.25643 handle_command_reply: tid=0 2023-08-11T15:49:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 client.25643 shutdown 2023-08-11T15:49:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 msgr2=0x55e0f0a9d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e44099b90 tx=0x7f5e440ca020 comp rx=0 tx=0).stop 2023-08-11T15:49:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 shutdown_connections 2023-08-11T15:49:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e0f0afc3c0 0x55e0f0afe7b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0f0a93e80 0x55e0f0a9ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f0a8e630 0x55e0f0a9d0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 --2- 172.21.15.32:0/3032295437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0f0a8eff0 0x55e0f0a9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e30ff9700 0 client.25643 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 >> 172.21.15.32:0/3032295437 conn(0x55e0f09bb8f0 msgr2=0x55e0f0a8d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 shutdown_connections 2023-08-11T15:49:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 1 -- 172.21.15.32:0/3032295437 wait complete. 2023-08-11T15:49:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.130+0000 7f5e630a7b80 20 client.25643 trim_cache size 0 max 16384 2023-08-11T15:49:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:49:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:49:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:49:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:49:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f5e340a00c0 msgr2=0x7f5e340a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f5e340a00c0 0x7f5e340a2580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e4819e860 tx=0x7f5e38009260 comp rx=0 tx=0).stop 2023-08-11T15:49:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e481008a0 msgr2=0x7f5e481a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e481008a0 0x7f5e481a1e40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005910 tx=0x7f5e44004710 comp rx=0 tx=0).stop 2023-08-11T15:49:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 shutdown_connections 2023-08-11T15:49:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f5e340a00c0 0x7f5e340a2580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e481008a0 0x7f5e481a1e40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e481068b0 0x7f5e4810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 --2- 172.21.15.32:0/2188144773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e48101260 0x7f5e481a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 >> 172.21.15.32:0/2188144773 conn(0x7f5e480fc370 msgr2=0x7f5e4810bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 shutdown_connections 2023-08-11T15:49:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:38.131+0000 7f5e4f7ff700 1 -- 172.21.15.32:0/2188144773 wait complete. 2023-08-11T15:49:38.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:37 smithi134 ceph-mon[130447]: pgmap v94: 129 pgs: 129 active+clean; 44 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 389 op/s 2023-08-11T15:49:38.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:37 smithi032 ceph-mon[97103]: pgmap v94: 129 pgs: 129 active+clean; 44 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 389 op/s 2023-08-11T15:49:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:37 smithi192 ceph-mon[130806]: pgmap v94: 129 pgs: 129 active+clean; 44 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 389 op/s 2023-08-11T15:49:38.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 280767 13.43 MB/sec warmup 114 sec latency 207.557 ms 2023-08-11T15:49:39.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 283857 13.47 MB/sec warmup 115 sec latency 7.633 ms 2023-08-11T15:49:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:40 smithi032 ceph-mon[97103]: pgmap v95: 129 pgs: 129 active+clean; 46 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2023-08-11T15:49:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:40 smithi192 ceph-mon[130806]: pgmap v95: 129 pgs: 129 active+clean; 46 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2023-08-11T15:49:40.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:40 smithi134 ceph-mon[130447]: pgmap v95: 129 pgs: 129 active+clean; 46 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2023-08-11T15:49:40.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 286226 13.43 MB/sec warmup 116 sec latency 21.644 ms 2023-08-11T15:49:41.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 288539 13.43 MB/sec warmup 117 sec latency 13.897 ms 2023-08-11T15:49:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:42 smithi032 ceph-mon[97103]: pgmap v96: 129 pgs: 129 active+clean; 46 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2023-08-11T15:49:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:42 smithi192 ceph-mon[130806]: pgmap v96: 129 pgs: 129 active+clean; 46 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2023-08-11T15:49:42.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:42 smithi134 ceph-mon[130447]: pgmap v96: 129 pgs: 129 active+clean; 46 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2023-08-11T15:49:42.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 291471 13.46 MB/sec warmup 118 sec latency 17.819 ms 2023-08-11T15:49:43.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 294579 13.50 MB/sec warmup 119 sec latency 65.772 ms 2023-08-11T15:49:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:44 smithi032 ceph-mon[97103]: pgmap v97: 129 pgs: 129 active+clean; 40 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 436 op/s 2023-08-11T15:49:44.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:44 smithi192 ceph-mon[130806]: pgmap v97: 129 pgs: 129 active+clean; 40 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 436 op/s 2023-08-11T15:49:44.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:44 smithi134 ceph-mon[130447]: pgmap v97: 129 pgs: 129 active+clean; 40 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 436 op/s 2023-08-11T15:49:45.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 299046 13.90 MB/sec execute 1 sec latency 121.949 ms 2023-08-11T15:49:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:46 smithi032 ceph-mon[97103]: pgmap v98: 129 pgs: 129 active+clean; 40 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 352 op/s 2023-08-11T15:49:46.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:46 smithi192 ceph-mon[130806]: pgmap v98: 129 pgs: 129 active+clean; 40 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 352 op/s 2023-08-11T15:49:46.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:46 smithi134 ceph-mon[130447]: pgmap v98: 129 pgs: 129 active+clean; 40 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 352 op/s 2023-08-11T15:49:46.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 301977 15.53 MB/sec execute 2 sec latency 32.016 ms 2023-08-11T15:49:47.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 305028 16.16 MB/sec execute 3 sec latency 8.969 ms 2023-08-11T15:49:48.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:47 smithi134 ceph-mon[130447]: pgmap v99: 129 pgs: 129 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 429 op/s 2023-08-11T15:49:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:47 smithi032 ceph-mon[97103]: pgmap v99: 129 pgs: 129 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 429 op/s 2023-08-11T15:49:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:47 smithi192 ceph-mon[130806]: pgmap v99: 129 pgs: 129 active+clean; 46 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 429 op/s 2023-08-11T15:49:48.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 307443 14.30 MB/sec execute 4 sec latency 29.480 ms 2023-08-11T15:49:49.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 309403 13.95 MB/sec execute 5 sec latency 106.478 ms 2023-08-11T15:49:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:50 smithi032 ceph-mon[97103]: pgmap v100: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 368 op/s 2023-08-11T15:49:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:50 smithi192 ceph-mon[130806]: pgmap v100: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 368 op/s 2023-08-11T15:49:50.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:50 smithi134 ceph-mon[130447]: pgmap v100: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 368 op/s 2023-08-11T15:49:50.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 312435 14.69 MB/sec execute 6 sec latency 26.441 ms 2023-08-11T15:49:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.294+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3540468279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 msgr2=0x7fe24810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.294+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3540468279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24810d770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe24000b0a0 tx=0x7fe24000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.295+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3540468279 shutdown_connections 2023-08-11T15:49:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.295+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3540468279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2481068b0 0x7fe248106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.295+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3540468279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24810d770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.295+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3540468279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2481008a0 0x7fe248100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.295+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3540468279 >> 172.21.15.32:0/3540468279 conn(0x7fe2480fc370 msgr2=0x7fe2480fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.295+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3540468279 shutdown_connections 2023-08-11T15:49:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.295+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3540468279 wait complete. 2023-08-11T15:49:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.296+0000 7fe24d1ff700 1 Processor -- start 2023-08-11T15:49:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.296+0000 7fe24d1ff700 1 -- start start 2023-08-11T15:49:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24d1ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2481008a0 0x7fe24819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24d1ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24d1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2481068b0 0x7fe248198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24d1ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe248198f70 con 0x7fe2481008a0 2023-08-11T15:49:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24d1ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2481990e0 con 0x7fe248101260 2023-08-11T15:49:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24d1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2481992a0 con 0x7fe2481068b0 2023-08-11T15:49:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44678/0 (socket says 172.21.15.32:44678) 2023-08-11T15:49:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24659c700 1 -- 172.21.15.32:0/3452779885 learned_addr learned my addr 172.21.15.32:0/3452779885 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24759e700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2481068b0 0x7fe248198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe246d9d700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2481008a0 0x7fe24819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24759e700 1 -- 172.21.15.32:0/3452779885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 msgr2=0x7fe24819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24759e700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24759e700 1 -- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2481008a0 msgr2=0x7fe24819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24759e700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2481008a0 0x7fe24819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24759e700 1 -- 172.21.15.32:0/3452779885 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c009520 con 0x7fe2481068b0 2023-08-11T15:49:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.297+0000 7fe24659c700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:49:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.298+0000 7fe24759e700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2481068b0 0x7fe248198900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe240009fd0 tx=0x7fe2400075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.298+0000 7fe22ffff700 1 -- 172.21.15.32:0/3452779885 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000e040 con 0x7fe2481068b0 2023-08-11T15:49:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.298+0000 7fe22ffff700 1 -- 172.21.15.32:0/3452779885 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe240004e60 con 0x7fe2481068b0 2023-08-11T15:49:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.298+0000 7fe22ffff700 1 -- 172.21.15.32:0/3452779885 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2400052a0 con 0x7fe2481068b0 2023-08-11T15:49:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.298+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe240009d00 con 0x7fe2481068b0 2023-08-11T15:49:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.298+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2481a58b0 con 0x7fe2481068b0 2023-08-11T15:49:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.302+0000 7fe22ffff700 1 -- 172.21.15.32:0/3452779885 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fe240005400 con 0x7fe2481068b0 2023-08-11T15:49:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.304+0000 7fe22ffff700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe2300a00c0 0x7fe2300a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.305+0000 7fe22ffff700 1 -- 172.21.15.32:0/3452779885 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe2400cd060 con 0x7fe2481068b0 2023-08-11T15:49:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.305+0000 7fe260bbeb80 1 Processor -- start 2023-08-11T15:49:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.305+0000 7fe246d9d700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe2300a00c0 0x7fe2300a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.305+0000 7fe260bbeb80 1 -- start start 2023-08-11T15:49:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee682a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67d800 0x55f6ee689420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee5cd5d0 con 0x55f6ee67d800 2023-08-11T15:49:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee5caef0 con 0x55f6ee682690 2023-08-11T15:49:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee5c67e0 con 0x55f6ee67ce40 2023-08-11T15:49:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe24759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67d800 0x55f6ee689420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe24659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe24759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67d800 0x55f6ee689420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43794/0 (socket says 172.21.15.32:43794) 2023-08-11T15:49:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe24759e700 1 -- 172.21.15.32:0/129350441 learned_addr learned my addr 172.21.15.32:0/129350441 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.306+0000 7fe246d9d700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe2300a00c0 0x7fe2300a2580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe23800dad0 tx=0x7fe23800d4b0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe246d9d700 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee682a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe24659c700 1 -- 172.21.15.32:0/129350441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 msgr2=0x55f6ee682a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe24659c700 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee682a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe24659c700 1 -- 172.21.15.32:0/129350441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67d800 msgr2=0x55f6ee689420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe24659c700 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67d800 0x55f6ee689420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe24659c700 1 -- 172.21.15.32:0/129350441 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ee4465d0 con 0x55f6ee67ce40 2023-08-11T15:49:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe24759e700 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67d800 0x55f6ee689420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:49:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe24659c700 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d2c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0117e0 tx=0x7fe23c011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df276c6fade61dc4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe22d7fa700 1 -- 172.21.15.32:0/129350441 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c004020 con 0x55f6ee67ce40 2023-08-11T15:49:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe22d7fa700 1 -- 172.21.15.32:0/129350441 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe23c003680 con 0x55f6ee67ce40 2023-08-11T15:49:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.307+0000 7fe22d7fa700 1 -- 172.21.15.32:0/129350441 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00f760 con 0x55f6ee67ce40 2023-08-11T15:49:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 -- 172.21.15.32:0/129350441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 msgr2=0x55f6ee67d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d2c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0117e0 tx=0x7fe23c011af0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 -- 172.21.15.32:0/129350441 shutdown_connections 2023-08-11T15:49:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67d800 0x55f6ee689420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee682a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/129350441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d2c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 -- 172.21.15.32:0/129350441 >> 172.21.15.32:0/129350441 conn(0x55f6ee5a18f0 msgr2=0x55f6ee66ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.308+0000 7fe260bbeb80 1 -- 172.21.15.32:0/129350441 shutdown_connections 2023-08-11T15:49:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.309+0000 7fe260bbeb80 1 -- 172.21.15.32:0/129350441 wait complete. 2023-08-11T15:49:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.309+0000 7fe260bbeb80 1 -- start start 2023-08-11T15:49:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 1 Processor -- start 2023-08-11T15:49:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee68c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67d800 0x55f6ee68c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee6864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee564a60 con 0x55f6ee682690 2023-08-11T15:49:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee564760 con 0x55f6ee67d800 2023-08-11T15:49:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee4488c0 con 0x55f6ee67ce40 2023-08-11T15:49:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe260bbeb80 10 client.0 _ll_register_callbacks cb 0x55f6ee567a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee68c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe24659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67d800 0x55f6ee68c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee68c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34550/0 (socket says 172.21.15.32:34550) 2023-08-11T15:49:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.310+0000 7fe24659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67d800 0x55f6ee68c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44688/0 (socket says 172.21.15.32:44688) 2023-08-11T15:49:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe246d9d700 1 -- 172.21.15.32:0/149035643 learned_addr learned my addr 172.21.15.32:0/149035643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24759e700 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee6864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24659c700 1 -- 172.21.15.32:0/149035643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 msgr2=0x55f6ee68c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24659c700 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee68c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24659c700 1 -- 172.21.15.32:0/149035643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 msgr2=0x55f6ee6864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24659c700 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee6864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24659c700 1 -- 172.21.15.32:0/149035643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2481a58b0 con 0x55f6ee67d800 2023-08-11T15:49:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24759e700 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee6864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:49:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe24659c700 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67d800 0x55f6ee68c6c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe23c01b040 tx=0x7fe23c003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c2f063595e232c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe22aff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:49:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe22aff5700 1 -- 172.21.15.32:0/149035643 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c004340 con 0x55f6ee67d800 2023-08-11T15:49:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.311+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6ee6677e0 con 0x55f6ee67d800 2023-08-11T15:49:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe22aff5700 1 -- 172.21.15.32:0/149035643 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe23c0044a0 con 0x55f6ee67d800 2023-08-11T15:49:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe22aff5700 1 -- 172.21.15.32:0/149035643 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe23c005040 con 0x55f6ee67d800 2023-08-11T15:49:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 10 client.25655 fetch_fsmap learned FSMap version 27 2023-08-11T15:49:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe23c011440 con 0x55f6ee67d800 2023-08-11T15:49:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe22aff5700 1 -- 172.21.15.32:0/149035643 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6ee6677e0 con 0x55f6ee67d800 2023-08-11T15:49:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe22aff5700 1 -- 172.21.15.32:0/149035643 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe23c00f540 con 0x55f6ee67d800 2023-08-11T15:49:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 10 client.25655 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:49:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 10 client.25655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 20 client.25655 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee696680 0x55f6ee698aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 4 client.25655 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:49:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.312+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6ee643480 con 0x55f6ee696680 2023-08-11T15:49:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.313+0000 7fe246d9d700 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee696680 0x55f6ee698aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.313+0000 7fe246d9d700 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee696680 0x55f6ee698aa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51297407ac60100 server_cookie=8989904d269d75e1 in_seq=0 out_seq=0 2023-08-11T15:49:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.313+0000 7fe22aff5700 10 client.25655 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe22aff5700 1 -- 172.21.15.32:0/149035643 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f6ee643480 con 0x55f6ee696680 2023-08-11T15:49:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe22aff5700 10 client.25655 handle_command_reply: tid=0 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 client.25655 shutdown 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67d800 msgr2=0x55f6ee68c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67d800 0x55f6ee68c6c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe23c01b040 tx=0x7fe23c003cf0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 shutdown_connections 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee696680 0x55f6ee698aa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee6864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe22aff5700 0 client.25655 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67ce40 0x55f6ee68c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/149035643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67d800 0x55f6ee68c6c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 >> 172.21.15.32:0/149035643 conn(0x55f6ee5a18f0 msgr2=0x55f6ee66e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 shutdown_connections 2023-08-11T15:49:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 1 -- 172.21.15.32:0/149035643 wait complete. 2023-08-11T15:49:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.314+0000 7fe260bbeb80 20 client.25655 trim_cache size 0 max 16384 2023-08-11T15:49:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.330+0000 7fe260bbeb80 1 Processor -- start 2023-08-11T15:49:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.330+0000 7fe260bbeb80 1 -- start start 2023-08-11T15:49:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee66ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee67fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee692fa0 con 0x55f6ee682690 2023-08-11T15:49:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee66f290 con 0x55f6ee67ce40 2023-08-11T15:49:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee6800e0 con 0x55f6ee67d800 2023-08-11T15:49:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee66ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee66ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34558/0 (socket says 172.21.15.32:34558) 2023-08-11T15:49:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44692/0 (socket says 172.21.15.32:44692) 2023-08-11T15:49:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 -- 172.21.15.32:0/530666946 learned_addr learned my addr 172.21.15.32:0/530666946 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24759e700 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee67fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 -- 172.21.15.32:0/530666946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67ce40 msgr2=0x55f6ee67d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 -- 172.21.15.32:0/530666946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 msgr2=0x55f6ee67fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee67fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24659c700 1 -- 172.21.15.32:0/530666946 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ee3f2d80 con 0x55f6ee67d800 2023-08-11T15:49:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe246d9d700 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:49:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.331+0000 7fe24759e700 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee67fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:49:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.332+0000 7fe24659c700 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee66ed50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe23c011730 tx=0x7fe23c0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a8da43be824ac4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.332+0000 7fe22dffb700 1 -- 172.21.15.32:0/530666946 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00f540 con 0x55f6ee67d800 2023-08-11T15:49:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.332+0000 7fe22dffb700 1 -- 172.21.15.32:0/530666946 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe23c00f6a0 con 0x55f6ee67d800 2023-08-11T15:49:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.332+0000 7fe22dffb700 1 -- 172.21.15.32:0/530666946 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c019990 con 0x55f6ee67d800 2023-08-11T15:49:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 -- 172.21.15.32:0/530666946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 msgr2=0x55f6ee66ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee66ed50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe23c011730 tx=0x7fe23c0092e0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 -- 172.21.15.32:0/530666946 shutdown_connections 2023-08-11T15:49:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee682690 0x55f6ee67fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee67ce40 0x55f6ee67d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/530666946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee66ed50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 -- 172.21.15.32:0/530666946 >> 172.21.15.32:0/530666946 conn(0x55f6ee5a18f0 msgr2=0x55f6ee6707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 -- 172.21.15.32:0/530666946 shutdown_connections 2023-08-11T15:49:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 -- 172.21.15.32:0/530666946 wait complete. 2023-08-11T15:49:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 -- start start 2023-08-11T15:49:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.333+0000 7fe260bbeb80 1 Processor -- start 2023-08-11T15:49:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67ce40 0x55f6ee689bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee68a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe260bbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee6e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee68a8a0 con 0x55f6ee67ce40 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee6e9870 con 0x55f6ee682690 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe260bbeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ee6e99e0 con 0x55f6ee67d800 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe260bbeb80 10 client.0 _ll_register_callbacks cb 0x7fe240096110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67ce40 0x55f6ee689bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe24659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee68a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe24759e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee6e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe24659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee68a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:34560/0 (socket says 172.21.15.32:34560) 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe24759e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee6e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44706/0 (socket says 172.21.15.32:44706) 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe246d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67ce40 0x55f6ee689bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43834/0 (socket says 172.21.15.32:43834) 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.334+0000 7fe24659c700 1 -- 172.21.15.32:0/2479336618 learned_addr learned my addr 172.21.15.32:0/2479336618 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe24759e700 1 -- 172.21.15.32:0/2479336618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 msgr2=0x55f6ee68a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe24759e700 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee68a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe24759e700 1 -- 172.21.15.32:0/2479336618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67ce40 msgr2=0x55f6ee689bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:49:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe24759e700 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67ce40 0x55f6ee689bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe24759e700 1 -- 172.21.15.32:0/2479336618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23800f850 con 0x55f6ee682690 2023-08-11T15:49:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe24659c700 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee68a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:49:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe24759e700 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee6e9330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe240096990 tx=0x7fe24000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f4db1704e66e99 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:49:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe22aff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:49:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe22aff5700 1 -- 172.21.15.32:0/2479336618 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24009ab90 con 0x55f6ee682690 2023-08-11T15:49:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe22aff5700 1 -- 172.21.15.32:0/2479336618 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe24009a490 con 0x55f6ee682690 2023-08-11T15:49:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe23c003680 con 0x55f6ee682690 2023-08-11T15:49:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe22aff5700 1 -- 172.21.15.32:0/2479336618 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe24009e2c0 con 0x55f6ee682690 2023-08-11T15:49:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe260bbeb80 10 client.25661 fetch_fsmap learned FSMap version 27 2023-08-11T15:49:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe23c011440 con 0x55f6ee682690 2023-08-11T15:49:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.335+0000 7fe22aff5700 1 -- 172.21.15.32:0/2479336618 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe23c003680 con 0x55f6ee682690 2023-08-11T15:49:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe22aff5700 1 -- 172.21.15.32:0/2479336618 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe240099980 con 0x55f6ee682690 2023-08-11T15:49:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe260bbeb80 10 client.25661 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:49:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe260bbeb80 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:49:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe260bbeb80 20 client.25661 populate_metadata read hostname 'smithi032' 2023-08-11T15:49:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee6eaf10 0x55f6ee6ed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:49:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe260bbeb80 4 client.25661 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:49:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f6ee5bf7c0 con 0x55f6ee6eaf10 2023-08-11T15:49:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.336+0000 7fe246d9d700 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee6eaf10 0x55f6ee6ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:49:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.337+0000 7fe246d9d700 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee6eaf10 0x55f6ee6ed300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6069b527fa470e server_cookie=840972db8083d8ee in_seq=0 out_seq=0 2023-08-11T15:49:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.337+0000 7fe22aff5700 10 client.25661 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.655+0000 7fe22aff5700 1 -- 172.21.15.32:0/2479336618 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f6ee5bf7c0 con 0x55f6ee6eaf10 2023-08-11T15:49:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.655+0000 7fe22aff5700 10 client.25661 handle_command_reply: tid=0 2023-08-11T15:49:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.655+0000 7fe260bbeb80 1 client.25661 shutdown 2023-08-11T15:49:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 msgr2=0x55f6ee6e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee6e9330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe240096990 tx=0x7fe24000b3b0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 shutdown_connections 2023-08-11T15:49:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f6ee6eaf10 0x55f6ee6ed300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ee67ce40 0x55f6ee689bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6ee682690 0x55f6ee6e9330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 --2- 172.21.15.32:0/2479336618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6ee67d800 0x55f6ee68a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.656+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 >> 172.21.15.32:0/2479336618 conn(0x55f6ee5a18f0 msgr2=0x55f6ee670720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.657+0000 7fe22aff5700 0 client.25661 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:49:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.657+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 shutdown_connections 2023-08-11T15:49:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.657+0000 7fe260bbeb80 1 -- 172.21.15.32:0/2479336618 wait complete. 2023-08-11T15:49:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.657+0000 7fe260bbeb80 20 client.25661 trim_cache size 0 max 16384 2023-08-11T15:49:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:49:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:49:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:49:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:49:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe2300a00c0 msgr2=0x7fe2300a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe2300a00c0 0x7fe2300a2580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe23800dad0 tx=0x7fe23800d4b0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2481068b0 msgr2=0x7fe248198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:49:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2481068b0 0x7fe248198900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe240009fd0 tx=0x7fe2400075c0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 shutdown_connections 2023-08-11T15:49:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe2300a00c0 0x7fe2300a2580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2481008a0 0x7fe24819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2481068b0 0x7fe248198900 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 --2- 172.21.15.32:0/3452779885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe248101260 0x7fe24819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:49:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.658+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 >> 172.21.15.32:0/3452779885 conn(0x7fe2480fc370 msgr2=0x7fe24810cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:49:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.659+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 shutdown_connections 2023-08-11T15:49:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:49:51.659+0000 7fe24d1ff700 1 -- 172.21.15.32:0/3452779885 wait complete. 2023-08-11T15:49:51.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 313981 13.40 MB/sec execute 7 sec latency 206.943 ms 2023-08-11T15:49:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:52 smithi032 ceph-mon[97103]: pgmap v101: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 354 op/s 2023-08-11T15:49:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:52 smithi192 ceph-mon[130806]: pgmap v101: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 354 op/s 2023-08-11T15:49:52.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:52 smithi134 ceph-mon[130447]: pgmap v101: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 354 op/s 2023-08-11T15:49:52.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 316427 13.48 MB/sec execute 8 sec latency 26.078 ms 2023-08-11T15:49:53.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:53 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:49:53.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:53 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:49:53.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:53 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:49:53.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 317114 12.51 MB/sec execute 9 sec latency 294.639 ms 2023-08-11T15:49:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:54 smithi032 ceph-mon[97103]: pgmap v102: 129 pgs: 129 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 622 op/s 2023-08-11T15:49:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:54 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:49:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:54 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:49:54.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:54 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:49:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:54 smithi192 ceph-mon[130806]: pgmap v102: 129 pgs: 129 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 622 op/s 2023-08-11T15:49:54.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:54 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:49:54.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:54 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:49:54.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:54 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:49:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:54 smithi134 ceph-mon[130447]: pgmap v102: 129 pgs: 129 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 622 op/s 2023-08-11T15:49:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:54 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:49:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:54 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:49:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:54 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:49:54.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 319255 12.50 MB/sec execute 10 sec latency 47.199 ms 2023-08-11T15:49:55.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 321725 12.22 MB/sec execute 11 sec latency 11.354 ms 2023-08-11T15:49:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:56 smithi032 ceph-mon[97103]: pgmap v103: 129 pgs: 129 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 353 op/s 2023-08-11T15:49:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:56 smithi192 ceph-mon[130806]: pgmap v103: 129 pgs: 129 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 353 op/s 2023-08-11T15:49:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:56 smithi134 ceph-mon[130447]: pgmap v103: 129 pgs: 129 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 353 op/s 2023-08-11T15:49:56.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323974 12.32 MB/sec execute 12 sec latency 14.960 ms 2023-08-11T15:49:57.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 326923 12.69 MB/sec execute 13 sec latency 6.731 ms 2023-08-11T15:49:58.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:49:57 smithi134 ceph-mon[130447]: pgmap v104: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 558 op/s 2023-08-11T15:49:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:49:57 smithi032 ceph-mon[97103]: pgmap v104: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 558 op/s 2023-08-11T15:49:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:49:57 smithi192 ceph-mon[130806]: pgmap v104: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 558 op/s 2023-08-11T15:49:58.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 329347 12.68 MB/sec execute 14 sec latency 157.809 ms 2023-08-11T15:49:59.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 330829 12.32 MB/sec execute 15 sec latency 252.375 ms 2023-08-11T15:50:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:00 smithi032 ceph-mon[97103]: pgmap v105: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 540 op/s 2023-08-11T15:50:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:00 smithi032 ceph-mon[97103]: overall HEALTH_OK 2023-08-11T15:50:00.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:00 smithi192 ceph-mon[130806]: pgmap v105: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 540 op/s 2023-08-11T15:50:00.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:00 smithi192 ceph-mon[130806]: overall HEALTH_OK 2023-08-11T15:50:00.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:00 smithi134 ceph-mon[130447]: pgmap v105: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 540 op/s 2023-08-11T15:50:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:00 smithi134 ceph-mon[130447]: overall HEALTH_OK 2023-08-11T15:50:00.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 332171 11.90 MB/sec execute 16 sec latency 61.284 ms 2023-08-11T15:50:01.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 333724 11.93 MB/sec execute 17 sec latency 83.321 ms 2023-08-11T15:50:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:02 smithi032 ceph-mon[97103]: pgmap v106: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2023-08-11T15:50:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:02 smithi192 ceph-mon[130806]: pgmap v106: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2023-08-11T15:50:02.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:02 smithi134 ceph-mon[130447]: pgmap v106: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2023-08-11T15:50:02.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 336600 12.02 MB/sec execute 18 sec latency 10.734 ms 2023-08-11T15:50:03.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 338868 11.98 MB/sec execute 19 sec latency 9.450 ms 2023-08-11T15:50:04.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:04 smithi032 ceph-mon[97103]: pgmap v107: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 737 op/s 2023-08-11T15:50:04.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:04 smithi192 ceph-mon[130806]: pgmap v107: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 737 op/s 2023-08-11T15:50:04.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:04 smithi134 ceph-mon[130447]: pgmap v107: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 737 op/s 2023-08-11T15:50:04.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 341402 12.10 MB/sec execute 20 sec latency 7.441 ms 2023-08-11T15:50:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.818+0000 7f268e9ff700 1 -- 172.21.15.32:0/1342871642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 msgr2=0x7f2688100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.818+0000 7f268e9ff700 1 --2- 172.21.15.32:0/1342871642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f2688100d30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f267c009990 tx=0x7f267c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.819+0000 7f268e9ff700 1 -- 172.21.15.32:0/1342871642 shutdown_connections 2023-08-11T15:50:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.819+0000 7f268e9ff700 1 --2- 172.21.15.32:0/1342871642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26881068c0 0x7f2688106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.819+0000 7f268e9ff700 1 --2- 172.21.15.32:0/1342871642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2688101270 0x7f268810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.819+0000 7f268e9ff700 1 --2- 172.21.15.32:0/1342871642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f2688100d30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.819+0000 7f268e9ff700 1 -- 172.21.15.32:0/1342871642 >> 172.21.15.32:0/1342871642 conn(0x7f26880fc380 msgr2=0x7f26880fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.819+0000 7f268e9ff700 1 -- 172.21.15.32:0/1342871642 shutdown_connections 2023-08-11T15:50:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.819+0000 7f268e9ff700 1 -- 172.21.15.32:0/1342871642 wait complete. 2023-08-11T15:50:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.820+0000 7f268e9ff700 1 Processor -- start 2023-08-11T15:50:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.820+0000 7f268e9ff700 1 -- start start 2023-08-11T15:50:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f268e9ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f268819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f268e9ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2688101270 0x7f268819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f268e9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26881068c0 0x7f26881989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f268e9ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2688199050 con 0x7f2688101270 2023-08-11T15:50:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f268e9ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881991c0 con 0x7f26881008b0 2023-08-11T15:50:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f268e9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2688199380 con 0x7f26881068c0 2023-08-11T15:50:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f2687fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f268819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f2687fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f268819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:45394/0 (socket says 172.21.15.32:45394) 2023-08-11T15:50:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f2687fff700 1 -- 172.21.15.32:0/4122170928 learned_addr learned my addr 172.21.15.32:0/4122170928 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.821+0000 7f26877fe700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2688101270 0x7f268819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f2687fff700 1 -- 172.21.15.32:0/4122170928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26881068c0 msgr2=0x7f26881989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f2687fff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26881068c0 0x7f26881989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f2687fff700 1 -- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2688101270 msgr2=0x7f268819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f2687fff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2688101270 0x7f268819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f2687fff700 1 -- 172.21.15.32:0/4122170928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c0095f0 con 0x7f26881008b0 2023-08-11T15:50:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f2687fff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f268819eb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9a0 tx=0x7f267800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f26857fa700 1 -- 172.21.15.32:0/4122170928 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678009790 con 0x7f26881008b0 2023-08-11T15:50:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f26857fa700 1 -- 172.21.15.32:0/4122170928 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f267800cb50 con 0x7f26881008b0 2023-08-11T15:50:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2688199660 con 0x7f26881008b0 2023-08-11T15:50:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f26857fa700 1 -- 172.21.15.32:0/4122170928 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678018680 con 0x7f26881008b0 2023-08-11T15:50:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.822+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26881a5b70 con 0x7f26881008b0 2023-08-11T15:50:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.827+0000 7f26857fa700 1 -- 172.21.15.32:0/4122170928 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f267800ccc0 con 0x7f26881008b0 2023-08-11T15:50:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.830+0000 7f26857fa700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f26700a0140 0x7f26700a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.830+0000 7f26a239eb80 1 Processor -- start 2023-08-11T15:50:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.830+0000 7f26857fa700 1 -- 172.21.15.32:0/4122170928 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2678014070 con 0x7f26881008b0 2023-08-11T15:50:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.835+0000 7f26a239eb80 1 -- start start 2023-08-11T15:50:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.835+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea52260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.835+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea4cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.835+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea58c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.836+0000 7f26a239eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739e9a55d0 con 0x55739ea4c630 2023-08-11T15:50:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.836+0000 7f26a239eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739e9a2ef0 con 0x55739ea51e80 2023-08-11T15:50:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.836+0000 7f26a239eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739e99e7e0 con 0x55739ea4cff0 2023-08-11T15:50:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.836+0000 7f2687fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea52260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.837+0000 7f2687fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea52260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:45398/0 (socket says 172.21.15.32:45398) 2023-08-11T15:50:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.837+0000 7f2687fff700 1 -- 172.21.15.32:0/520911472 learned_addr learned my addr 172.21.15.32:0/520911472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.838+0000 7f268cf9c700 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea58c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.839+0000 7f26877fe700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f26700a0140 0x7f26700a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.839+0000 7f268cf9c700 1 -- 172.21.15.32:0/520911472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 msgr2=0x55739ea52260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.839+0000 7f268cf9c700 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea52260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.839+0000 7f268cf9c700 1 -- 172.21.15.32:0/520911472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 msgr2=0x55739ea4cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.839+0000 7f268cf9c700 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea4cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.839+0000 7f268cf9c700 1 -- 172.21.15.32:0/520911472 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739e81e5d0 con 0x55739ea4cff0 2023-08-11T15:50:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.840+0000 7f268cf9c700 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea58c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2680009f30 tx=0x7f268000e740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6675744638751f93 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.841+0000 7f266e7fc700 1 -- 172.21.15.32:0/520911472 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680004db0 con 0x55739ea4cff0 2023-08-11T15:50:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.841+0000 7f266e7fc700 1 -- 172.21.15.32:0/520911472 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f268000f040 con 0x55739ea4cff0 2023-08-11T15:50:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.841+0000 7f266e7fc700 1 -- 172.21.15.32:0/520911472 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680017d70 con 0x55739ea4cff0 2023-08-11T15:50:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.843+0000 7f26a239eb80 1 -- 172.21.15.32:0/520911472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 msgr2=0x55739ea58c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.843+0000 7f26a239eb80 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea58c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2680009f30 tx=0x7f268000e740 comp rx=0 tx=0).stop 2023-08-11T15:50:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.843+0000 7f26877fe700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f26700a0140 0x7f26700a2600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2688105ff0 tx=0x7f267c00ba70 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.846+0000 7f26a239eb80 1 -- 172.21.15.32:0/520911472 shutdown_connections 2023-08-11T15:50:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.846+0000 7f26a239eb80 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea58c10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.846+0000 7f26a239eb80 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea52260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.846+0000 7f26a239eb80 1 --2- 172.21.15.32:0/520911472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea4cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.846+0000 7f26a239eb80 1 -- 172.21.15.32:0/520911472 >> 172.21.15.32:0/520911472 conn(0x55739e9798f0 msgr2=0x55739ea46e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.849+0000 7f26a239eb80 1 -- 172.21.15.32:0/520911472 shutdown_connections 2023-08-11T15:50:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.849+0000 7f26a239eb80 1 -- 172.21.15.32:0/520911472 wait complete. 2023-08-11T15:50:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.850+0000 7f26a239eb80 1 -- start start 2023-08-11T15:50:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.851+0000 7f26a239eb80 1 Processor -- start 2023-08-11T15:50:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.853+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea5ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.853+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.853+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea55d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.853+0000 7f26a239eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739e93ca60 con 0x55739ea4c630 2023-08-11T15:50:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.853+0000 7f26a239eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739e93c760 con 0x55739ea51e80 2023-08-11T15:50:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.853+0000 7f26a239eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739e8208c0 con 0x55739ea4cff0 2023-08-11T15:50:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.853+0000 7f26a239eb80 10 client.0 _ll_register_callbacks cb 0x55739e93fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.856+0000 7f2687fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea5ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.856+0000 7f2687fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea5ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49106/0 (socket says 172.21.15.32:49106) 2023-08-11T15:50:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.856+0000 7f2687fff700 1 -- 172.21.15.32:0/2795098567 learned_addr learned my addr 172.21.15.32:0/2795098567 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.857+0000 7f2687fff700 1 -- 172.21.15.32:0/2795098567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 msgr2=0x55739ea5bfa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:50:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.857+0000 7f2687fff700 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea5bfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.857+0000 7f2687fff700 1 -- 172.21.15.32:0/2795098567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 msgr2=0x55739ea55d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T15:50:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.857+0000 7f2687fff700 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea55d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.857+0000 7f2687fff700 1 -- 172.21.15.32:0/2795098567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268000e140 con 0x55739ea4c630 2023-08-11T15:50:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.857+0000 7f2687fff700 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea5ba60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f26780958d0 tx=0x7f2678099690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774725c1e2e4f2ac server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.857+0000 7f26677fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:50:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.858+0000 7f26677fe700 1 -- 172.21.15.32:0/2795098567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26780a0c70 con 0x55739ea4c630 2023-08-11T15:50:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.858+0000 7f26677fe700 1 -- 172.21.15.32:0/2795098567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2678098350 con 0x55739ea4c630 2023-08-11T15:50:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.858+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55739ea3f7e0 con 0x55739ea4c630 2023-08-11T15:50:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.858+0000 7f26677fe700 1 -- 172.21.15.32:0/2795098567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2678098510 con 0x55739ea4c630 2023-08-11T15:50:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.859+0000 7f26a239eb80 10 client.15657 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.859+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55739e7cad80 con 0x55739ea4c630 2023-08-11T15:50:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.859+0000 7f26677fe700 1 -- 172.21.15.32:0/2795098567 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55739ea3f7e0 con 0x55739ea4c630 2023-08-11T15:50:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.859+0000 7f26677fe700 1 -- 172.21.15.32:0/2795098567 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2678097f50 con 0x55739ea4c630 2023-08-11T15:50:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.859+0000 7f26a239eb80 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.859+0000 7f26a239eb80 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.860+0000 7f26a239eb80 20 client.15657 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.860+0000 7f26a239eb80 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739ea76f10 0x55739ea79360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.860+0000 7f26a239eb80 4 client.15657 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:50:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.860+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55739ea1b480 con 0x55739ea76f10 2023-08-11T15:50:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.869+0000 7f26877fe700 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739ea76f10 0x55739ea79360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.871+0000 7f26877fe700 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739ea76f10 0x55739ea79360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6861d1b8ba99bc25 server_cookie=4cf10c2c16e95e3d in_seq=0 out_seq=0 2023-08-11T15:50:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.871+0000 7f26677fe700 10 client.15657 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.872+0000 7f26677fe700 1 -- 172.21.15.32:0/2795098567 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55739ea1b480 con 0x55739ea76f10 2023-08-11T15:50:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.872+0000 7f26677fe700 10 client.15657 handle_command_reply: tid=0 2023-08-11T15:50:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.872+0000 7f26a239eb80 1 client.15657 shutdown 2023-08-11T15:50:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.872+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 msgr2=0x55739ea5ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.872+0000 7f26a239eb80 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea5ba60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f26780958d0 tx=0x7f2678099690 comp rx=0 tx=0).stop 2023-08-11T15:50:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 shutdown_connections 2023-08-11T15:50:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26a239eb80 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739ea76f10 0x55739ea79360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26a239eb80 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739ea5ba60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26a239eb80 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea51e80 0x55739ea55d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26a239eb80 1 --2- 172.21.15.32:0/2795098567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4cff0 0x55739ea5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 >> 172.21.15.32:0/2795098567 conn(0x55739e9798f0 msgr2=0x55739ea465d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26677fe700 0 client.15657 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.873+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 shutdown_connections 2023-08-11T15:50:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.874+0000 7f26a239eb80 1 -- 172.21.15.32:0/2795098567 wait complete. 2023-08-11T15:50:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.874+0000 7f26a239eb80 20 client.15657 trim_cache size 0 max 16384 2023-08-11T15:50:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 Processor -- start 2023-08-11T15:50:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 -- start start 2023-08-11T15:50:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4c630 0x55739ea539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739ea5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea51e80 0x55739ea46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ea573a0 con 0x55739ea51e80 2023-08-11T15:50:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ea48db0 con 0x55739ea4cff0 2023-08-11T15:50:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.890+0000 7f26a239eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ea3f570 con 0x55739ea4c630 2023-08-11T15:50:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739ea5b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f2687fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4c630 0x55739ea539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739ea5b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:45424/0 (socket says 172.21.15.32:45424) 2023-08-11T15:50:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 -- 172.21.15.32:0/3808991807 learned_addr learned my addr 172.21.15.32:0/3808991807 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f2687fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4c630 0x55739ea539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:58016/0 (socket says 172.21.15.32:58016) 2023-08-11T15:50:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f268cf9c700 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea51e80 0x55739ea46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 -- 172.21.15.32:0/3808991807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4c630 msgr2=0x55739ea539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4c630 0x55739ea539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 -- 172.21.15.32:0/3808991807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea51e80 msgr2=0x55739ea46d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea51e80 0x55739ea46d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 -- 172.21.15.32:0/3808991807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739e7b3c00 con 0x55739ea4cff0 2023-08-11T15:50:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f26877fe700 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739ea5b5c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f267c000c00 tx=0x7f267c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76912a980ec3d9cd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.891+0000 7f266effd700 1 -- 172.21.15.32:0/3808991807 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c02b040 con 0x55739ea4cff0 2023-08-11T15:50:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f266effd700 1 -- 172.21.15.32:0/3808991807 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f267c01e040 con 0x55739ea4cff0 2023-08-11T15:50:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f266effd700 1 -- 172.21.15.32:0/3808991807 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0047f0 con 0x55739ea4cff0 2023-08-11T15:50:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 -- 172.21.15.32:0/3808991807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 msgr2=0x55739ea5b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739ea5b5c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f267c000c00 tx=0x7f267c020000 comp rx=0 tx=0).stop 2023-08-11T15:50:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 -- 172.21.15.32:0/3808991807 shutdown_connections 2023-08-11T15:50:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea51e80 0x55739ea46d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea4c630 0x55739ea539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 --2- 172.21.15.32:0/3808991807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739ea5b5c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 -- 172.21.15.32:0/3808991807 >> 172.21.15.32:0/3808991807 conn(0x55739e9798f0 msgr2=0x55739ea4ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 -- 172.21.15.32:0/3808991807 shutdown_connections 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.892+0000 7f26a239eb80 1 -- 172.21.15.32:0/3808991807 wait complete. 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.893+0000 7f26a239eb80 1 -- start start 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.893+0000 7f26a239eb80 1 Processor -- start 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739eac2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739eac3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26a239eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea51e80 0x55739eac3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26a239eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739eac4360 con 0x55739ea4c630 2023-08-11T15:50:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26a239eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739eac44d0 con 0x55739ea4cff0 2023-08-11T15:50:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26a239eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739eac4690 con 0x55739ea51e80 2023-08-11T15:50:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26a239eb80 10 client.0 _ll_register_callbacks cb 0x7f2678098a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26877fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739eac3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26877fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739eac3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:45426/0 (socket says 172.21.15.32:45426) 2023-08-11T15:50:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f26877fe700 1 -- 172.21.15.32:0/1072051652 learned_addr learned my addr 172.21.15.32:0/1072051652 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f2687fff700 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739eac2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.894+0000 7f268cf9c700 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea51e80 0x55739eac3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26877fe700 1 -- 172.21.15.32:0/1072051652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea51e80 msgr2=0x55739eac3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26877fe700 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea51e80 0x55739eac3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26877fe700 1 -- 172.21.15.32:0/1072051652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 msgr2=0x55739eac2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26877fe700 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739eac2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26877fe700 1 -- 172.21.15.32:0/1072051652 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268810e080 con 0x55739ea4cff0 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26877fe700 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739eac3450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f267c026040 tx=0x7f267c005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff45f474965460ce server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26677fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:50:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26677fe700 1 -- 172.21.15.32:0/1072051652 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c02b070 con 0x55739ea4cff0 2023-08-11T15:50:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f268000f040 con 0x55739ea4cff0 2023-08-11T15:50:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26677fe700 1 -- 172.21.15.32:0/1072051652 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f267c0304b0 con 0x55739ea4cff0 2023-08-11T15:50:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26677fe700 1 -- 172.21.15.32:0/1072051652 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267c005450 con 0x55739ea4cff0 2023-08-11T15:50:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26a239eb80 10 client.25676 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.895+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55739e7f11f0 con 0x55739ea4cff0 2023-08-11T15:50:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26677fe700 1 -- 172.21.15.32:0/1072051652 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f268000f040 con 0x55739ea4cff0 2023-08-11T15:50:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26677fe700 1 -- 172.21.15.32:0/1072051652 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f267c00f4f0 con 0x55739ea4cff0 2023-08-11T15:50:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26a239eb80 10 client.25676 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26a239eb80 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26a239eb80 20 client.25676 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26a239eb80 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739eacba40 0x55739eacde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26a239eb80 4 client.25676 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:50:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55739e9977c0 con 0x55739eacba40 2023-08-11T15:50:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.896+0000 7f2687fff700 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739eacba40 0x55739eacde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.897+0000 7f2687fff700 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739eacba40 0x55739eacde60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6e17d9640af500 server_cookie=9b876ca1c1beb7e8 in_seq=0 out_seq=0 2023-08-11T15:50:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:04.897+0000 7f26677fe700 10 client.25676 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.135+0000 7f26677fe700 1 -- 172.21.15.32:0/1072051652 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55739e9977c0 con 0x55739eacba40 2023-08-11T15:50:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.135+0000 7f26677fe700 10 client.25676 handle_command_reply: tid=0 2023-08-11T15:50:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.135+0000 7f26a239eb80 1 client.25676 shutdown 2023-08-11T15:50:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.135+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 msgr2=0x55739eac3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.135+0000 7f26a239eb80 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739eac3450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f267c026040 tx=0x7f267c005bc0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 shutdown_connections 2023-08-11T15:50:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55739eacba40 0x55739eacde60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55739ea4c630 0x55739eac2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55739ea51e80 0x55739eac3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 --2- 172.21.15.32:0/1072051652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55739ea4cff0 0x55739eac3450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 >> 172.21.15.32:0/1072051652 conn(0x55739e9798f0 msgr2=0x55739ea4b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26677fe700 0 client.25676 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 shutdown_connections 2023-08-11T15:50:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.136+0000 7f26a239eb80 1 -- 172.21.15.32:0/1072051652 wait complete. 2023-08-11T15:50:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f26a239eb80 20 client.25676 trim_cache size 0 max 16384 2023-08-11T15:50:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:50:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:50:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:50:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:50:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f26700a0140 msgr2=0x7f26700a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f26700a0140 0x7f26700a2600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2688105ff0 tx=0x7f267c00ba70 comp rx=0 tx=0).stop 2023-08-11T15:50:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 msgr2=0x7f268819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f268819eb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9a0 tx=0x7f267800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 shutdown_connections 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f26700a0140 0x7f26700a2600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26881068c0 0x7f26881989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26881008b0 0x7f268819eb80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 --2- 172.21.15.32:0/4122170928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2688101270 0x7f268819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.137+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 >> 172.21.15.32:0/4122170928 conn(0x7f26880fc380 msgr2=0x7f26880fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.138+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 shutdown_connections 2023-08-11T15:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:05.138+0000 7f268e9ff700 1 -- 172.21.15.32:0/4122170928 wait complete. 2023-08-11T15:50:05.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 343883 12.18 MB/sec execute 21 sec latency 132.815 ms 2023-08-11T15:50:06.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:06 smithi032 ceph-mon[97103]: pgmap v108: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.3 MiB/s wr, 469 op/s 2023-08-11T15:50:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:06 smithi192 ceph-mon[130806]: pgmap v108: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.3 MiB/s wr, 469 op/s 2023-08-11T15:50:06.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:06 smithi134 ceph-mon[130447]: pgmap v108: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.3 MiB/s wr, 469 op/s 2023-08-11T15:50:06.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 346023 12.09 MB/sec execute 22 sec latency 62.795 ms 2023-08-11T15:50:07.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 348411 12.18 MB/sec execute 23 sec latency 12.652 ms 2023-08-11T15:50:08.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:07 smithi134 ceph-mon[130447]: pgmap v109: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 644 op/s 2023-08-11T15:50:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:07 smithi032 ceph-mon[97103]: pgmap v109: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 644 op/s 2023-08-11T15:50:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:07 smithi192 ceph-mon[130806]: pgmap v109: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 644 op/s 2023-08-11T15:50:08.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 351476 12.43 MB/sec execute 24 sec latency 13.242 ms 2023-08-11T15:50:09.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354090 12.45 MB/sec execute 25 sec latency 30.659 ms 2023-08-11T15:50:10.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:10 smithi032 ceph-mon[97103]: pgmap v110: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 480 op/s 2023-08-11T15:50:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:10 smithi192 ceph-mon[130806]: pgmap v110: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 480 op/s 2023-08-11T15:50:10.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:10 smithi134 ceph-mon[130447]: pgmap v110: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 480 op/s 2023-08-11T15:50:10.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 356475 12.43 MB/sec execute 26 sec latency 10.780 ms 2023-08-11T15:50:11.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 358984 12.49 MB/sec execute 27 sec latency 35.517 ms 2023-08-11T15:50:12.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:12 smithi032 ceph-mon[97103]: pgmap v111: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 421 op/s 2023-08-11T15:50:12.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:12 smithi192 ceph-mon[130806]: pgmap v111: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 421 op/s 2023-08-11T15:50:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:12 smithi134 ceph-mon[130447]: pgmap v111: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 421 op/s 2023-08-11T15:50:12.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 362032 12.71 MB/sec execute 28 sec latency 10.998 ms 2023-08-11T15:50:13.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 364691 12.69 MB/sec execute 29 sec latency 16.587 ms 2023-08-11T15:50:14.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:14 smithi032 ceph-mon[97103]: pgmap v112: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 504 op/s 2023-08-11T15:50:14.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:14 smithi192 ceph-mon[130806]: pgmap v112: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 504 op/s 2023-08-11T15:50:14.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:14 smithi134 ceph-mon[130447]: pgmap v112: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 504 op/s 2023-08-11T15:50:14.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 366978 12.68 MB/sec execute 30 sec latency 8.022 ms 2023-08-11T15:50:15.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 369503 12.72 MB/sec execute 31 sec latency 12.646 ms 2023-08-11T15:50:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:16 smithi032 ceph-mon[97103]: pgmap v113: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 298 op/s 2023-08-11T15:50:16.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:16 smithi192 ceph-mon[130806]: pgmap v113: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 298 op/s 2023-08-11T15:50:16.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:16 smithi134 ceph-mon[130447]: pgmap v113: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 298 op/s 2023-08-11T15:50:16.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 372669 12.91 MB/sec execute 32 sec latency 17.308 ms 2023-08-11T15:50:17.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 374305 12.69 MB/sec execute 33 sec latency 187.455 ms 2023-08-11T15:50:18.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:17 smithi134 ceph-mon[130447]: pgmap v114: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 372 op/s 2023-08-11T15:50:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 -- 172.21.15.32:0/1897530270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 msgr2=0x7fd82410c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 --2- 172.21.15.32:0/1897530270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd82410c670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b0a0 tx=0x7fd81c00b3b0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 -- 172.21.15.32:0/1897530270 shutdown_connections 2023-08-11T15:50:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 --2- 172.21.15.32:0/1897530270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd82410c670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 --2- 172.21.15.32:0/1897530270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8241057b0 0x7fd824105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 --2- 172.21.15.32:0/1897530270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8240ff7a0 0x7fd8240ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 -- 172.21.15.32:0/1897530270 >> 172.21.15.32:0/1897530270 conn(0x7fd8240fb290 msgr2=0x7fd8240fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 -- 172.21.15.32:0/1897530270 shutdown_connections 2023-08-11T15:50:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.295+0000 7fd828dff700 1 -- 172.21.15.32:0/1897530270 wait complete. 2023-08-11T15:50:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.296+0000 7fd828dff700 1 Processor -- start 2023-08-11T15:50:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.296+0000 7fd828dff700 1 -- start start 2023-08-11T15:50:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd828dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8240ff7a0 0x7fd824198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd828dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd8241991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd828dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8241057b0 0x7fd82419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd828dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82419cdb0 con 0x7fd8241057b0 2023-08-11T15:50:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd828dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82419cf20 con 0x7fd824100160 2023-08-11T15:50:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd828dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82419d0e0 con 0x7fd8240ff7a0 2023-08-11T15:50:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd8241991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd82259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8240ff7a0 0x7fd824198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd8241991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50628/0 (socket says 172.21.15.32:50628) 2023-08-11T15:50:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd82259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8240ff7a0 0x7fd824198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:47590/0 (socket says 172.21.15.32:47590) 2023-08-11T15:50:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 -- 172.21.15.32:0/3707673148 learned_addr learned my addr 172.21.15.32:0/3707673148 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd822d9d700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8241057b0 0x7fd82419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 -- 172.21.15.32:0/3707673148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8240ff7a0 msgr2=0x7fd824198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8240ff7a0 0x7fd824198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 -- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8241057b0 msgr2=0x7fd82419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8241057b0 0x7fd82419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd821d9b700 1 -- 172.21.15.32:0/3707673148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c009d00 con 0x7fd824100160 2023-08-11T15:50:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.297+0000 7fd82259c700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8240ff7a0 0x7fd824198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.298+0000 7fd821d9b700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd8241991f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd81800e9a0 tx=0x7fd81800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.298+0000 7fd8137fe700 1 -- 172.21.15.32:0/3707673148 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818009790 con 0x7fd824100160 2023-08-11T15:50:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.298+0000 7fd8137fe700 1 -- 172.21.15.32:0/3707673148 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd818010430 con 0x7fd824100160 2023-08-11T15:50:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.298+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82419d3c0 con 0x7fd824100160 2023-08-11T15:50:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.298+0000 7fd8137fe700 1 -- 172.21.15.32:0/3707673148 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818018680 con 0x7fd824100160 2023-08-11T15:50:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.298+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8241a5d20 con 0x7fd824100160 2023-08-11T15:50:18.307 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:17 smithi032 ceph-mon[97103]: pgmap v114: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 372 op/s 2023-08-11T15:50:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.302+0000 7fd8137fe700 1 -- 172.21.15.32:0/3707673148 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fd8180105a0 con 0x7fd824100160 2023-08-11T15:50:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.303+0000 7fd8137fe700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd80c0a00c0 0x7fd80c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.303+0000 7fd8137fe700 1 -- 172.21.15.32:0/3707673148 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fd818014070 con 0x7fd824100160 2023-08-11T15:50:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.303+0000 7fd82259c700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd80c0a00c0 0x7fd80c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 Processor -- start 2023-08-11T15:50:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 -- start start 2023-08-11T15:50:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d951260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd82259c700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd80c0a00c0 0x7fd80c0a2580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd814009fd0 tx=0x7fd814009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d94bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d957c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d8a45d0 con 0x55f48d94b630 2023-08-11T15:50:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d8a1ef0 con 0x55f48d950e80 2023-08-11T15:50:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d89d7e0 con 0x55f48d94bff0 2023-08-11T15:50:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd821d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d94bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd82259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d951260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd822d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d957c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d957c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:47594/0 (socket says 172.21.15.32:47594) 2023-08-11T15:50:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 -- 172.21.15.32:0/878334644 learned_addr learned my addr 172.21.15.32:0/878334644 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.304+0000 7fd82259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d951260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50636/0 (socket says 172.21.15.32:50636) 2023-08-11T15:50:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 -- 172.21.15.32:0/878334644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 msgr2=0x55f48d951260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d951260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 -- 172.21.15.32:0/878334644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 msgr2=0x55f48d94bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d94bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 -- 172.21.15.32:0/878334644 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f48d71d5d0 con 0x55f48d94bff0 2023-08-11T15:50:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd821d9b700 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d94bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd822d9d700 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d957c10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd81c015000 tx=0x7fd81c009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8544582c7f09913 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd810ff9700 1 -- 172.21.15.32:0/878334644 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00e040 con 0x55f48d94bff0 2023-08-11T15:50:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd810ff9700 1 -- 172.21.15.32:0/878334644 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81c00bd30 con 0x55f48d94bff0 2023-08-11T15:50:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.305+0000 7fd810ff9700 1 -- 172.21.15.32:0/878334644 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c01c6a0 con 0x55f48d94bff0 2023-08-11T15:50:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 -- 172.21.15.32:0/878334644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 msgr2=0x55f48d957c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d957c10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd81c015000 tx=0x7fd81c009450 comp rx=0 tx=0).stop 2023-08-11T15:50:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 -- 172.21.15.32:0/878334644 shutdown_connections 2023-08-11T15:50:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d957c10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d951260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/878334644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d94bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 -- 172.21.15.32:0/878334644 >> 172.21.15.32:0/878334644 conn(0x55f48d8788f0 msgr2=0x55f48d945e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 -- 172.21.15.32:0/878334644 shutdown_connections 2023-08-11T15:50:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.306+0000 7fd83c7dab80 1 -- 172.21.15.32:0/878334644 wait complete. 2023-08-11T15:50:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.307+0000 7fd83c7dab80 1 -- start start 2023-08-11T15:50:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.307+0000 7fd83c7dab80 1 Processor -- start 2023-08-11T15:50:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.307+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94b630 0x55f48d95a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.307+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d95aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.307+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d950e80 0x55f48d954c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d83ba60 con 0x55f48d950e80 2023-08-11T15:50:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d83b760 con 0x55f48d94b630 2023-08-11T15:50:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d71f8c0 con 0x55f48d94bff0 2023-08-11T15:50:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd83c7dab80 10 client.0 _ll_register_callbacks cb 0x55f48d83ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94b630 0x55f48d95a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94b630 0x55f48d95a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50652/0 (socket says 172.21.15.32:50652) 2023-08-11T15:50:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 -- 172.21.15.32:0/206599308 learned_addr learned my addr 172.21.15.32:0/206599308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd821d9b700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d95aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd822d9d700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d950e80 0x55f48d954c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 -- 172.21.15.32:0/206599308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 msgr2=0x55f48d95aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d95aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 -- 172.21.15.32:0/206599308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d950e80 msgr2=0x55f48d954c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d950e80 0x55f48d954c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 -- 172.21.15.32:0/206599308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8241a5d20 con 0x55f48d94b630 2023-08-11T15:50:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd821d9b700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d95aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.308+0000 7fd82259c700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94b630 0x55f48d95a970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd81400ebc0 tx=0x7fd8140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c336d458fc864575 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd809ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:50:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd809ffb700 1 -- 172.21.15.32:0/206599308 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814004460 con 0x55f48d94b630 2023-08-11T15:50:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f48d93e7e0 con 0x55f48d94b630 2023-08-11T15:50:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd809ffb700 1 -- 172.21.15.32:0/206599308 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fd81401f890 con 0x55f48d94b630 2023-08-11T15:50:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd809ffb700 1 -- 172.21.15.32:0/206599308 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd81401fa50 con 0x55f48d94b630 2023-08-11T15:50:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd83c7dab80 10 client.25688 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd82410cf70 con 0x55f48d94b630 2023-08-11T15:50:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.309+0000 7fd809ffb700 1 -- 172.21.15.32:0/206599308 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f48d93e7e0 con 0x55f48d94b630 2023-08-11T15:50:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd809ffb700 1 -- 172.21.15.32:0/206599308 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd814028440 con 0x55f48d94b630 2023-08-11T15:50:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd83c7dab80 10 client.25688 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd83c7dab80 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd83c7dab80 20 client.25688 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d964e40 0x55f48d967260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd83c7dab80 4 client.25688 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:50:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f48d91a480 con 0x55f48d964e40 2023-08-11T15:50:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd821d9b700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d964e40 0x55f48d967260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd821d9b700 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d964e40 0x55f48d967260 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b32d080dd9e65f server_cookie=429d89675edab7b3 in_seq=0 out_seq=0 2023-08-11T15:50:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.310+0000 7fd809ffb700 10 client.25688 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd809ffb700 1 -- 172.21.15.32:0/206599308 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f48d91a480 con 0x55f48d964e40 2023-08-11T15:50:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd809ffb700 10 client.25688 handle_command_reply: tid=0 2023-08-11T15:50:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 client.25688 shutdown 2023-08-11T15:50:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94b630 msgr2=0x55f48d95a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94b630 0x55f48d95a970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd81400ebc0 tx=0x7fd8140047c0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 shutdown_connections 2023-08-11T15:50:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d964e40 0x55f48d967260 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d950e80 0x55f48d954c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94b630 0x55f48d95a970 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd809ffb700 0 client.25688 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/206599308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94bff0 0x55f48d95aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.311+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 >> 172.21.15.32:0/206599308 conn(0x55f48d8788f0 msgr2=0x55f48d9455d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.312+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 shutdown_connections 2023-08-11T15:50:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.312+0000 7fd83c7dab80 1 -- 172.21.15.32:0/206599308 wait complete. 2023-08-11T15:50:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.312+0000 7fd83c7dab80 20 client.25688 trim_cache size 0 max 16384 2023-08-11T15:50:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 Processor -- start 2023-08-11T15:50:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 -- start start 2023-08-11T15:50:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d96bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94bff0 0x55f48d95a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d950e80 0x55f48d945d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d946a60 con 0x55f48d94b630 2023-08-11T15:50:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d947db0 con 0x55f48d94bff0 2023-08-11T15:50:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d946f70 con 0x55f48d950e80 2023-08-11T15:50:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd82259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d96bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd821d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94bff0 0x55f48d95a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd821d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94bff0 0x55f48d95a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50654/0 (socket says 172.21.15.32:50654) 2023-08-11T15:50:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd821d9b700 1 -- 172.21.15.32:0/1951169930 learned_addr learned my addr 172.21.15.32:0/1951169930 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd82259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d96bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46200/0 (socket says 172.21.15.32:46200) 2023-08-11T15:50:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.329+0000 7fd822d9d700 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d950e80 0x55f48d945d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd821d9b700 1 -- 172.21.15.32:0/1951169930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d950e80 msgr2=0x55f48d945d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd821d9b700 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d950e80 0x55f48d945d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd821d9b700 1 -- 172.21.15.32:0/1951169930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 msgr2=0x55f48d96bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd821d9b700 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d96bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd821d9b700 1 -- 172.21.15.32:0/1951169930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f48d6c9d80 con 0x55f48d94bff0 2023-08-11T15:50:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd82259c700 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d96bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd821d9b700 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94bff0 0x55f48d95a490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd818009230 tx=0x7fd818095390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2595d4eb094476c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd8117fa700 1 -- 172.21.15.32:0/1951169930 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8180a0d30 con 0x55f48d94bff0 2023-08-11T15:50:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd8117fa700 1 -- 172.21.15.32:0/1951169930 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8180972c0 con 0x55f48d94bff0 2023-08-11T15:50:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.330+0000 7fd8117fa700 1 -- 172.21.15.32:0/1951169930 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8180a08a0 con 0x55f48d94bff0 2023-08-11T15:50:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 -- 172.21.15.32:0/1951169930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94bff0 msgr2=0x55f48d95a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94bff0 0x55f48d95a490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd818009230 tx=0x7fd818095390 comp rx=0 tx=0).stop 2023-08-11T15:50:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 -- 172.21.15.32:0/1951169930 shutdown_connections 2023-08-11T15:50:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94b630 0x55f48d96bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d950e80 0x55f48d945d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/1951169930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d94bff0 0x55f48d95a490 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 -- 172.21.15.32:0/1951169930 >> 172.21.15.32:0/1951169930 conn(0x55f48d8788f0 msgr2=0x55f48d94ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 -- 172.21.15.32:0/1951169930 shutdown_connections 2023-08-11T15:50:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 -- 172.21.15.32:0/1951169930 wait complete. 2023-08-11T15:50:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 -- start start 2023-08-11T15:50:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.331+0000 7fd83c7dab80 1 Processor -- start 2023-08-11T15:50:18.339 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:17 smithi192 ceph-mon[130806]: pgmap v114: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 372 op/s 2023-08-11T15:50:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94b630 0x55f48d9acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94bff0 0x55f48d9ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd83c7dab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d9b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d9b8060 con 0x55f48d94bff0 2023-08-11T15:50:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d9b81a0 con 0x55f48d950e80 2023-08-11T15:50:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd83c7dab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48d9b82e0 con 0x55f48d94b630 2023-08-11T15:50:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd83c7dab80 10 client.0 _ll_register_callbacks cb 0x7fd81400e980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd82259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94b630 0x55f48d9acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd82259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94b630 0x55f48d9acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:47626/0 (socket says 172.21.15.32:47626) 2023-08-11T15:50:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd82259c700 1 -- 172.21.15.32:0/2869654576 learned_addr learned my addr 172.21.15.32:0/2869654576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd822d9d700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d9b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.332+0000 7fd821d9b700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94bff0 0x55f48d9ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd82259c700 1 -- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 msgr2=0x55f48d9b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd82259c700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d9b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd82259c700 1 -- 172.21.15.32:0/2869654576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94bff0 msgr2=0x55f48d9ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd82259c700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94bff0 0x55f48d9ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd82259c700 1 -- 172.21.15.32:0/2869654576 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82410cf70 con 0x55f48d94b630 2023-08-11T15:50:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd821d9b700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94bff0 0x55f48d9ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:50:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd82259c700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94b630 0x55f48d9acc90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd81400efd0 tx=0x7fd81400fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d62c8f6eb64aaf44 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd809ffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:50:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd809ffb700 1 -- 172.21.15.32:0/2869654576 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81401f6e0 con 0x55f48d94b630 2023-08-11T15:50:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd81c00bd30 con 0x55f48d94b630 2023-08-11T15:50:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd809ffb700 1 -- 172.21.15.32:0/2869654576 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fd81401f840 con 0x55f48d94b630 2023-08-11T15:50:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd809ffb700 1 -- 172.21.15.32:0/2869654576 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd81401fa00 con 0x55f48d94b630 2023-08-11T15:50:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd83c7dab80 10 client.25693 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.333+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd82419d3c0 con 0x55f48d94b630 2023-08-11T15:50:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd809ffb700 1 -- 172.21.15.32:0/2869654576 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81c00bd30 con 0x55f48d94b630 2023-08-11T15:50:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd809ffb700 1 -- 172.21.15.32:0/2869654576 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd81400f3c0 con 0x55f48d94b630 2023-08-11T15:50:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd83c7dab80 10 client.25693 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd83c7dab80 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd83c7dab80 20 client.25693 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d9b9a00 0x55f48d9bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd83c7dab80 4 client.25693 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:50:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f48d8967c0 con 0x55f48d9b9a00 2023-08-11T15:50:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.334+0000 7fd821d9b700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d9b9a00 0x55f48d9bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.335+0000 7fd821d9b700 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d9b9a00 0x55f48d9bbdf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582a5ab1a8a8ae7d server_cookie=9c40c43a4f25143b in_seq=0 out_seq=0 2023-08-11T15:50:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.335+0000 7fd809ffb700 10 client.25693 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:18.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd809ffb700 1 -- 172.21.15.32:0/2869654576 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f48d8967c0 con 0x55f48d9b9a00 2023-08-11T15:50:18.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd809ffb700 10 client.25693 handle_command_reply: tid=0 2023-08-11T15:50:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 client.25693 shutdown 2023-08-11T15:50:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94b630 msgr2=0x55f48d9acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94b630 0x55f48d9acc90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd81400efd0 tx=0x7fd81400fe60 comp rx=0 tx=0).stop 2023-08-11T15:50:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 shutdown_connections 2023-08-11T15:50:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f48d9b9a00 0x55f48d9bbdf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f48d950e80 0x55f48d9b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f48d94b630 0x55f48d9acc90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 --2- 172.21.15.32:0/2869654576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f48d94bff0 0x55f48d9ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.603+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 >> 172.21.15.32:0/2869654576 conn(0x55f48d8788f0 msgr2=0x55f48d9544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.605+0000 7fd809ffb700 0 client.25693 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.606+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 shutdown_connections 2023-08-11T15:50:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.607+0000 7fd83c7dab80 1 -- 172.21.15.32:0/2869654576 wait complete. 2023-08-11T15:50:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.608+0000 7fd83c7dab80 20 client.25693 trim_cache size 0 max 16384 2023-08-11T15:50:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:50:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:50:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:50:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:50:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd80c0a00c0 msgr2=0x7fd80c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd80c0a00c0 0x7fd80c0a2580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd814009fd0 tx=0x7fd814009260 comp rx=0 tx=0).stop 2023-08-11T15:50:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 msgr2=0x7fd8241991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd8241991f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd81800e9a0 tx=0x7fd81800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 shutdown_connections 2023-08-11T15:50:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd80c0a00c0 0x7fd80c0a2580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8241057b0 0x7fd82419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8240ff7a0 0x7fd824198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 --2- 172.21.15.32:0/3707673148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd824100160 0x7fd8241991f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 >> 172.21.15.32:0/3707673148 conn(0x7fd8240fb290 msgr2=0x7fd82410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 shutdown_connections 2023-08-11T15:50:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:18.609+0000 7fd828dff700 1 -- 172.21.15.32:0/3707673148 wait complete. 2023-08-11T15:50:18.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 376218 12.71 MB/sec execute 34 sec latency 167.992 ms 2023-08-11T15:50:19.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 378609 12.59 MB/sec execute 35 sec latency 37.276 ms 2023-08-11T15:50:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:20 smithi032 ceph-mon[97103]: pgmap v115: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2023-08-11T15:50:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:20 smithi192 ceph-mon[130806]: pgmap v115: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2023-08-11T15:50:20.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:20 smithi134 ceph-mon[130447]: pgmap v115: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2023-08-11T15:50:20.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 380699 12.60 MB/sec execute 36 sec latency 17.253 ms 2023-08-11T15:50:21.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 383546 12.73 MB/sec execute 37 sec latency 9.827 ms 2023-08-11T15:50:22.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:22 smithi032 ceph-mon[97103]: pgmap v116: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 162 op/s 2023-08-11T15:50:22.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:22 smithi192 ceph-mon[130806]: pgmap v116: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 162 op/s 2023-08-11T15:50:22.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:22 smithi134 ceph-mon[130447]: pgmap v116: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 162 op/s 2023-08-11T15:50:22.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 385923 12.68 MB/sec execute 38 sec latency 86.629 ms 2023-08-11T15:50:23.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388206 12.70 MB/sec execute 39 sec latency 20.692 ms 2023-08-11T15:50:24.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:24 smithi032 ceph-mon[97103]: pgmap v117: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2023-08-11T15:50:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:24 smithi192 ceph-mon[130806]: pgmap v117: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2023-08-11T15:50:24.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:24 smithi134 ceph-mon[130447]: pgmap v117: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2023-08-11T15:50:24.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 390582 12.73 MB/sec execute 40 sec latency 242.293 ms 2023-08-11T15:50:25.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 393133 12.72 MB/sec execute 41 sec latency 83.585 ms 2023-08-11T15:50:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:26 smithi032 ceph-mon[97103]: pgmap v118: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 314 op/s 2023-08-11T15:50:26.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:26 smithi192 ceph-mon[130806]: pgmap v118: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 314 op/s 2023-08-11T15:50:26.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:26 smithi134 ceph-mon[130447]: pgmap v118: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 314 op/s 2023-08-11T15:50:26.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 395286 12.70 MB/sec execute 42 sec latency 43.689 ms 2023-08-11T15:50:27.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 397903 12.73 MB/sec execute 43 sec latency 40.367 ms 2023-08-11T15:50:28.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:27 smithi134 ceph-mon[130447]: pgmap v119: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 373 op/s 2023-08-11T15:50:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:27 smithi032 ceph-mon[97103]: pgmap v119: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 373 op/s 2023-08-11T15:50:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:27 smithi192 ceph-mon[130806]: pgmap v119: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 373 op/s 2023-08-11T15:50:28.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 400679 12.77 MB/sec execute 44 sec latency 72.920 ms 2023-08-11T15:50:29.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 401593 12.60 MB/sec execute 45 sec latency 425.258 ms 2023-08-11T15:50:30.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:30 smithi032 ceph-mon[97103]: pgmap v120: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 325 op/s 2023-08-11T15:50:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:30 smithi192 ceph-mon[130806]: pgmap v120: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 325 op/s 2023-08-11T15:50:30.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:30 smithi134 ceph-mon[130447]: pgmap v120: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 325 op/s 2023-08-11T15:50:30.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 403454 12.49 MB/sec execute 46 sec latency 126.322 ms 2023-08-11T15:50:31.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 405186 12.47 MB/sec execute 47 sec latency 24.358 ms 2023-08-11T15:50:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 -- 172.21.15.32:0/2532186851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 msgr2=0x7f3768106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 --2- 172.21.15.32:0/2532186851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f3768106cb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3764009920 tx=0x7f3764009c30 comp rx=0 tx=0).stop 2023-08-11T15:50:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 -- 172.21.15.32:0/2532186851 shutdown_connections 2023-08-11T15:50:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 --2- 172.21.15.32:0/2532186851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37680686d0 0x7f376810d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 --2- 172.21.15.32:0/2532186851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f3768106cb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 --2- 172.21.15.32:0/2532186851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3768067d10 0x7f3768068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 -- 172.21.15.32:0/2532186851 >> 172.21.15.32:0/2532186851 conn(0x7f3768075250 msgr2=0x7f3768075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 -- 172.21.15.32:0/2532186851 shutdown_connections 2023-08-11T15:50:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.772+0000 7f376f9ff700 1 -- 172.21.15.32:0/2532186851 wait complete. 2023-08-11T15:50:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.774+0000 7f376f9ff700 1 Processor -- start 2023-08-11T15:50:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.774+0000 7f376f9ff700 1 -- start start 2023-08-11T15:50:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376f9ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3768067d10 0x7f37680ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376f9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37680686d0 0x7f3768100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376f9ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f37681a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376f9ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37681a5250 con 0x7f3768067d10 2023-08-11T15:50:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376f9ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37681a5390 con 0x7f37681068d0 2023-08-11T15:50:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376f9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37681a5500 con 0x7f37680686d0 2023-08-11T15:50:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f37681a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f37681a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:38420/0 (socket says 172.21.15.32:38420) 2023-08-11T15:50:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 -- 172.21.15.32:0/1829287512 learned_addr learned my addr 172.21.15.32:0/1829287512 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376cf9a700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37680686d0 0x7f3768100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376d79b700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3768067d10 0x7f37680ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 -- 172.21.15.32:0/1829287512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37680686d0 msgr2=0x7f3768100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37680686d0 0x7f3768100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 -- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3768067d10 msgr2=0x7f37680ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3768067d10 0x7f37680ffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.775+0000 7f376df9c700 1 -- 172.21.15.32:0/1829287512 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37580095f0 con 0x7f37681068d0 2023-08-11T15:50:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.776+0000 7f376df9c700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f37681a4d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f376000ef30 tx=0x7f376000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.777+0000 7f375e7fc700 1 -- 172.21.15.32:0/1829287512 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760018070 con 0x7f37681068d0 2023-08-11T15:50:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.777+0000 7f375e7fc700 1 -- 172.21.15.32:0/1829287512 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376000f040 con 0x7f37681068d0 2023-08-11T15:50:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.777+0000 7f375e7fc700 1 -- 172.21.15.32:0/1829287512 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376000cdb0 con 0x7f37681068d0 2023-08-11T15:50:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.777+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3764009520 con 0x7f37681068d0 2023-08-11T15:50:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.777+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37681a5ab0 con 0x7f37681068d0 2023-08-11T15:50:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.782+0000 7f375e7fc700 1 -- 172.21.15.32:0/1829287512 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f3760003a00 con 0x7f37681068d0 2023-08-11T15:50:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.783+0000 7f37832e1b80 1 Processor -- start 2023-08-11T15:50:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f37832e1b80 1 -- start start 2023-08-11T15:50:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4b003690 0x556b4b003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4affe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4affe800 0x556b4b00a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f37832e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4af4e5d0 con 0x556b4b003690 2023-08-11T15:50:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f37832e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4af4bef0 con 0x556b4affe800 2023-08-11T15:50:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f37832e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4af477e0 con 0x556b4affde40 2023-08-11T15:50:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.785+0000 7f375e7fc700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f37540a01c0 0x7f37540a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f375e7fc700 1 -- 172.21.15.32:0/1829287512 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f37600cb4f0 con 0x7f37681068d0 2023-08-11T15:50:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376cf9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4affe800 0x556b4b00a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376cf9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4affe800 0x556b4b00a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:38426/0 (socket says 172.21.15.32:38426) 2023-08-11T15:50:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376cf9a700 1 -- 172.21.15.32:0/3266019458 learned_addr learned my addr 172.21.15.32:0/3266019458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376d79b700 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4affe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376df9c700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f37540a01c0 0x7f37540a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376cf9a700 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4b003690 0x556b4b003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376d79b700 1 -- 172.21.15.32:0/3266019458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4affe800 msgr2=0x556b4b00a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376d79b700 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4affe800 0x556b4b00a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376d79b700 1 -- 172.21.15.32:0/3266019458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4b003690 msgr2=0x556b4b003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376d79b700 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4b003690 0x556b4b003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.786+0000 7f376d79b700 1 -- 172.21.15.32:0/3266019458 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b4adc75d0 con 0x556b4affde40 2023-08-11T15:50:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.787+0000 7f376d79b700 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4affe2c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3764009230 tx=0x7f3764009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f57a30e27c7588f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.787+0000 7f376df9c700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f37540a01c0 0x7f37540a2680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f376000ef00 tx=0x7f3760009040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.787+0000 7f37537fe700 1 -- 172.21.15.32:0/3266019458 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376401c040 con 0x556b4affde40 2023-08-11T15:50:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.787+0000 7f37537fe700 1 -- 172.21.15.32:0/3266019458 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376400fdb0 con 0x556b4affde40 2023-08-11T15:50:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.787+0000 7f37537fe700 1 -- 172.21.15.32:0/3266019458 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764005040 con 0x556b4affde40 2023-08-11T15:50:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.788+0000 7f37832e1b80 1 -- 172.21.15.32:0/3266019458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 msgr2=0x556b4affe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.788+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4affe2c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3764009230 tx=0x7f3764009400 comp rx=0 tx=0).stop 2023-08-11T15:50:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.788+0000 7f37832e1b80 1 -- 172.21.15.32:0/3266019458 shutdown_connections 2023-08-11T15:50:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.788+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4b003690 0x556b4b003a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.788+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4affe800 0x556b4b00a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.788+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3266019458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4affe2c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.788+0000 7f37832e1b80 1 -- 172.21.15.32:0/3266019458 >> 172.21.15.32:0/3266019458 conn(0x556b4af228f0 msgr2=0x556b4afefe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.789+0000 7f37832e1b80 1 -- 172.21.15.32:0/3266019458 shutdown_connections 2023-08-11T15:50:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.789+0000 7f37832e1b80 1 -- 172.21.15.32:0/3266019458 wait complete. 2023-08-11T15:50:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.789+0000 7f37832e1b80 1 -- start start 2023-08-11T15:50:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.790+0000 7f37832e1b80 1 Processor -- start 2023-08-11T15:50:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b00d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b0074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f37832e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4aee5a60 con 0x556b4affde40 2023-08-11T15:50:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f37832e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4aee5760 con 0x556b4b003690 2023-08-11T15:50:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f37832e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4adc98c0 con 0x556b4affe800 2023-08-11T15:50:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f37832e1b80 10 client.0 _ll_register_callbacks cb 0x556b4aee8a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38042/0 (socket says 172.21.15.32:38042) 2023-08-11T15:50:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 -- 172.21.15.32:0/2890786037 learned_addr learned my addr 172.21.15.32:0/2890786037 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376cf9a700 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b00d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 -- 172.21.15.32:0/2890786037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 msgr2=0x556b4b0074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T15:50:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b0074c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 -- 172.21.15.32:0/2890786037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 msgr2=0x556b4b00d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b00d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.791+0000 7f376d79b700 1 -- 172.21.15.32:0/2890786037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376810e020 con 0x556b4affe800 2023-08-11T15:50:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f376cf9a700 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b00d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:50:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f376d79b700 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00d710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3764000c00 tx=0x7f3764005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679bdcb4b4ffdc32 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f3750ff9700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:50:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f3750ff9700 1 -- 172.21.15.32:0/2890786037 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376401c040 con 0x556b4affe800 2023-08-11T15:50:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b4afe87e0 con 0x556b4affe800 2023-08-11T15:50:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f3750ff9700 1 -- 172.21.15.32:0/2890786037 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3764005040 con 0x556b4affe800 2023-08-11T15:50:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f3750ff9700 1 -- 172.21.15.32:0/2890786037 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f376400f870 con 0x556b4affe800 2023-08-11T15:50:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f37832e1b80 10 client.25705 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.792+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556b4ad73d80 con 0x556b4affe800 2023-08-11T15:50:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f3750ff9700 1 -- 172.21.15.32:0/2890786037 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b4afe87e0 con 0x556b4affe800 2023-08-11T15:50:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f3750ff9700 1 -- 172.21.15.32:0/2890786037 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3764029040 con 0x556b4affe800 2023-08-11T15:50:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f37832e1b80 10 client.25705 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f37832e1b80 10 client.25705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f37832e1b80 20 client.25705 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f37832e1b80 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b0171f0 0x556b4b019610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f37832e1b80 4 client.25705 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:50:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b4afc4480 con 0x556b4b0171f0 2023-08-11T15:50:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.793+0000 7f376cf9a700 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b0171f0 0x556b4b019610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.794+0000 7f376cf9a700 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b0171f0 0x556b4b019610 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bca5422318a680b server_cookie=3522cb54aa1bbdba in_seq=0 out_seq=0 2023-08-11T15:50:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.794+0000 7f3750ff9700 10 client.25705 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.794+0000 7f3750ff9700 1 -- 172.21.15.32:0/2890786037 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x556b4afc4480 con 0x556b4b0171f0 2023-08-11T15:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.794+0000 7f3750ff9700 10 client.25705 handle_command_reply: tid=0 2023-08-11T15:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 client.25705 shutdown 2023-08-11T15:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 msgr2=0x556b4b00d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00d710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3764000c00 tx=0x7f3764005b00 comp rx=0 tx=0).stop 2023-08-11T15:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 shutdown_connections 2023-08-11T15:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b0171f0 0x556b4b019610 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b00d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b0074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 --2- 172.21.15.32:0/2890786037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00d710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f3750ff9700 0 client.25705 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 >> 172.21.15.32:0/2890786037 conn(0x556b4af228f0 msgr2=0x556b4afef5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 shutdown_connections 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 1 -- 172.21.15.32:0/2890786037 wait complete. 2023-08-11T15:50:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.795+0000 7f37832e1b80 20 client.25705 trim_cache size 0 max 16384 2023-08-11T15:50:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.811+0000 7f37832e1b80 1 Processor -- start 2023-08-11T15:50:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.811+0000 7f37832e1b80 1 -- start start 2023-08-11T15:50:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b01dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b007e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f37832e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4b005a20 con 0x556b4affde40 2023-08-11T15:50:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f37832e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4b0022c0 con 0x556b4b003690 2023-08-11T15:50:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f37832e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4b008390 con 0x556b4affe800 2023-08-11T15:50:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38048/0 (socket says 172.21.15.32:38048) 2023-08-11T15:50:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376cf9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b01dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 -- 172.21.15.32:0/1326011625 learned_addr learned my addr 172.21.15.32:0/1326011625 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376cf9a700 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b007e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 -- 172.21.15.32:0/1326011625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 msgr2=0x556b4b007e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b007e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 -- 172.21.15.32:0/1326011625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 msgr2=0x556b4b01dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b01dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.812+0000 7f376d79b700 1 -- 172.21.15.32:0/1326011625 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b4ad5cc00 con 0x556b4affe800 2023-08-11T15:50:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.813+0000 7f376d79b700 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00ccf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3764009c70 tx=0x7f3764004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b1e9a0b1fd6e61 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.813+0000 7f3753fff700 1 -- 172.21.15.32:0/1326011625 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376401c040 con 0x556b4affe800 2023-08-11T15:50:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.813+0000 7f3753fff700 1 -- 172.21.15.32:0/1326011625 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376400f870 con 0x556b4affe800 2023-08-11T15:50:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.813+0000 7f3753fff700 1 -- 172.21.15.32:0/1326011625 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764005470 con 0x556b4affe800 2023-08-11T15:50:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 -- 172.21.15.32:0/1326011625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 msgr2=0x556b4b00ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00ccf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3764009c70 tx=0x7f3764004f80 comp rx=0 tx=0).stop 2023-08-11T15:50:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 -- 172.21.15.32:0/1326011625 shutdown_connections 2023-08-11T15:50:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affde40 0x556b4b01dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b007e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 --2- 172.21.15.32:0/1326011625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affe800 0x556b4b00ccf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 -- 172.21.15.32:0/1326011625 >> 172.21.15.32:0/1326011625 conn(0x556b4af228f0 msgr2=0x556b4afec740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 -- 172.21.15.32:0/1326011625 shutdown_connections 2023-08-11T15:50:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 -- 172.21.15.32:0/1326011625 wait complete. 2023-08-11T15:50:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.814+0000 7f37832e1b80 1 -- start start 2023-08-11T15:50:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 1 Processor -- start 2023-08-11T15:50:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4b00a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affe800 0x556b4b00a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b00aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4b069ff0 con 0x556b4affe800 2023-08-11T15:50:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4b06a160 con 0x556b4b003690 2023-08-11T15:50:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4b06a320 con 0x556b4affde40 2023-08-11T15:50:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f37832e1b80 10 client.0 _ll_register_callbacks cb 0x7f3758011520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f376cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4b00a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f376cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4b00a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38062/0 (socket says 172.21.15.32:38062) 2023-08-11T15:50:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f376cf9a700 1 -- 172.21.15.32:0/3217576128 learned_addr learned my addr 172.21.15.32:0/3217576128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.815+0000 7f376cf9a700 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b00aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f376d79b700 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affe800 0x556b4b00a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f376cf9a700 1 -- 172.21.15.32:0/3217576128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 msgr2=0x556b4b00a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f376cf9a700 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4b00a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f376cf9a700 1 -- 172.21.15.32:0/3217576128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affe800 msgr2=0x556b4b00a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f376cf9a700 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affe800 0x556b4b00a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f376cf9a700 1 -- 172.21.15.32:0/3217576128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b4ad73d80 con 0x556b4b003690 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f376cf9a700 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b00aea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3758011700 tx=0x7f3758009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85e94f0f191cf18 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.816+0000 7f3750ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f3750ff9700 1 -- 172.21.15.32:0/3217576128 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758016040 con 0x556b4b003690 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f376400fdb0 con 0x556b4b003690 2023-08-11T15:50:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f3750ff9700 1 -- 172.21.15.32:0/3217576128 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f3758022e20 con 0x556b4b003690 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f3750ff9700 1 -- 172.21.15.32:0/3217576128 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3758004030 con 0x556b4b003690 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 10 client.25709 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556b4ad9a1f0 con 0x556b4b003690 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f3750ff9700 1 -- 172.21.15.32:0/3217576128 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376400fdb0 con 0x556b4b003690 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f3750ff9700 1 -- 172.21.15.32:0/3217576128 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3758011820 con 0x556b4b003690 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 10 client.25709 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 20 client.25709 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b06ba60 0x556b4b06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 4 client.25709 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:50:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.817+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b4af407c0 con 0x556b4b06ba60 2023-08-11T15:50:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.818+0000 7f376d79b700 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b06ba60 0x556b4b06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.818+0000 7f376d79b700 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b06ba60 0x556b4b06de80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3262eb3b3238d2 server_cookie=3d60a577b29cfef2 in_seq=0 out_seq=0 2023-08-11T15:50:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:31.818+0000 7f3750ff9700 10 client.25709 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.088+0000 7f3750ff9700 1 -- 172.21.15.32:0/3217576128 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b4af407c0 con 0x556b4b06ba60 2023-08-11T15:50:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.088+0000 7f3750ff9700 10 client.25709 handle_command_reply: tid=0 2023-08-11T15:50:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.088+0000 7f37832e1b80 1 client.25709 shutdown 2023-08-11T15:50:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 msgr2=0x556b4b00aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b00aea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3758011700 tx=0x7f3758009310 comp rx=0 tx=0).stop 2023-08-11T15:50:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 shutdown_connections 2023-08-11T15:50:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x556b4b06ba60 0x556b4b06de80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b4b003690 0x556b4b00aea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b4affde40 0x556b4b00a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 --2- 172.21.15.32:0/3217576128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b4affe800 0x556b4b00a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f3750ff9700 0 client.25709 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.089+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 >> 172.21.15.32:0/3217576128 conn(0x556b4af228f0 msgr2=0x556b4b0026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.090+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 shutdown_connections 2023-08-11T15:50:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.090+0000 7f37832e1b80 1 -- 172.21.15.32:0/3217576128 wait complete. 2023-08-11T15:50:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.090+0000 7f37832e1b80 20 client.25709 trim_cache size 0 max 16384 2023-08-11T15:50:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:50:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:50:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:50:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:50:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f37540a01c0 msgr2=0x7f37540a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f37540a01c0 0x7f37540a2680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f376000ef00 tx=0x7f3760009040 comp rx=0 tx=0).stop 2023-08-11T15:50:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 msgr2=0x7f37681a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f37681a4d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f376000ef30 tx=0x7f376000c3a0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 shutdown_connections 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f37540a01c0 0x7f37540a2680 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3768067d10 0x7f37680ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37681068d0 0x7f37681a4d10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 --2- 172.21.15.32:0/1829287512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37680686d0 0x7f3768100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 >> 172.21.15.32:0/1829287512 conn(0x7f3768075250 msgr2=0x7f37680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.091+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 shutdown_connections 2023-08-11T15:50:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:32.092+0000 7f376f9ff700 1 -- 172.21.15.32:0/1829287512 wait complete. 2023-08-11T15:50:32.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:32 smithi032 ceph-mon[97103]: pgmap v121: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 319 op/s 2023-08-11T15:50:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:32 smithi192 ceph-mon[130806]: pgmap v121: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 319 op/s 2023-08-11T15:50:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:32 smithi134 ceph-mon[130447]: pgmap v121: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 319 op/s 2023-08-11T15:50:32.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 407695 12.50 MB/sec execute 48 sec latency 146.968 ms 2023-08-11T15:50:33.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 409810 12.45 MB/sec execute 49 sec latency 23.763 ms 2023-08-11T15:50:34.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:34 smithi032 ceph-mon[97103]: pgmap v122: 129 pgs: 129 active+clean; 35 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 715 op/s 2023-08-11T15:50:34.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:34 smithi192 ceph-mon[130806]: pgmap v122: 129 pgs: 129 active+clean; 35 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 715 op/s 2023-08-11T15:50:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:34 smithi134 ceph-mon[130447]: pgmap v122: 129 pgs: 129 active+clean; 35 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 715 op/s 2023-08-11T15:50:34.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 411770 12.47 MB/sec execute 50 sec latency 97.498 ms 2023-08-11T15:50:35.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 413367 12.34 MB/sec execute 51 sec latency 61.927 ms 2023-08-11T15:50:36.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:36 smithi032 ceph-mon[97103]: pgmap v123: 129 pgs: 129 active+clean; 35 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 481 op/s 2023-08-11T15:50:36.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:36 smithi192 ceph-mon[130806]: pgmap v123: 129 pgs: 129 active+clean; 35 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 481 op/s 2023-08-11T15:50:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:36 smithi134 ceph-mon[130447]: pgmap v123: 129 pgs: 129 active+clean; 35 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 481 op/s 2023-08-11T15:50:36.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 415795 12.37 MB/sec execute 52 sec latency 6.388 ms 2023-08-11T15:50:37.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 418879 12.49 MB/sec execute 53 sec latency 25.812 ms 2023-08-11T15:50:38.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:37 smithi134 ceph-mon[130447]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 633 op/s 2023-08-11T15:50:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:37 smithi032 ceph-mon[97103]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 633 op/s 2023-08-11T15:50:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:37 smithi192 ceph-mon[130806]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 633 op/s 2023-08-11T15:50:38.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 421592 12.49 MB/sec execute 54 sec latency 6.543 ms 2023-08-11T15:50:39.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 423994 12.48 MB/sec execute 55 sec latency 17.623 ms 2023-08-11T15:50:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:40 smithi032 ceph-mon[97103]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 608 op/s 2023-08-11T15:50:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:40 smithi192 ceph-mon[130806]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 608 op/s 2023-08-11T15:50:40.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:40 smithi134 ceph-mon[130447]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 608 op/s 2023-08-11T15:50:40.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 426479 12.52 MB/sec execute 56 sec latency 31.010 ms 2023-08-11T15:50:41.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 429582 12.62 MB/sec execute 57 sec latency 11.557 ms 2023-08-11T15:50:42.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:42 smithi032 ceph-mon[97103]: pgmap v126: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.1 MiB/s wr, 583 op/s 2023-08-11T15:50:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:42 smithi192 ceph-mon[130806]: pgmap v126: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.1 MiB/s wr, 583 op/s 2023-08-11T15:50:42.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:42 smithi134 ceph-mon[130447]: pgmap v126: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.1 MiB/s wr, 583 op/s 2023-08-11T15:50:42.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 432390 12.63 MB/sec execute 58 sec latency 13.034 ms 2023-08-11T15:50:43.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 434425 12.61 MB/sec execute 59 sec latency 30.149 ms 2023-08-11T15:50:44.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:44 smithi032 ceph-mon[97103]: pgmap v127: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.8 MiB/s wr, 666 op/s 2023-08-11T15:50:44.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:44 smithi192 ceph-mon[130806]: pgmap v127: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.8 MiB/s wr, 666 op/s 2023-08-11T15:50:44.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:44 smithi134 ceph-mon[130447]: pgmap v127: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.8 MiB/s wr, 666 op/s 2023-08-11T15:50:44.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 436838 12.63 MB/sec execute 60 sec latency 113.637 ms 2023-08-11T15:50:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 -- 172.21.15.32:0/44348897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 msgr2=0x7f19500fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 --2- 172.21.15.32:0/44348897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 0x7f19500fff00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f19380099c0 tx=0x7f1938009cd0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 -- 172.21.15.32:0/44348897 shutdown_connections 2023-08-11T15:50:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 --2- 172.21.15.32:0/44348897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1950105840 0x7f1950105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 --2- 172.21.15.32:0/44348897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 0x7f195010c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 --2- 172.21.15.32:0/44348897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 0x7f19500fff00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 -- 172.21.15.32:0/44348897 >> 172.21.15.32:0/44348897 conn(0x7f1950075250 msgr2=0x7f1950075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 -- 172.21.15.32:0/44348897 shutdown_connections 2023-08-11T15:50:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.254+0000 7f1954dff700 1 -- 172.21.15.32:0/44348897 wait complete. 2023-08-11T15:50:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.255+0000 7f1954dff700 1 Processor -- start 2023-08-11T15:50:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1954dff700 1 -- start start 2023-08-11T15:50:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1954dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 0x7f195019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1954dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 0x7f195019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1954dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1950105840 0x7f19501968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1954dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1950197090 con 0x7f1950105840 2023-08-11T15:50:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1954dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1950197200 con 0x7f1950100440 2023-08-11T15:50:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1954dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19501973c0 con 0x7f19500ffa80 2023-08-11T15:50:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1947fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 0x7f195019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1947fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 0x7f195019d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39534/0 (socket says 172.21.15.32:39534) 2023-08-11T15:50:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f1947fff700 1 -- 172.21.15.32:0/3830703143 learned_addr learned my addr 172.21.15.32:0/3830703143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.256+0000 7f194ed9d700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1950105840 0x7f19501968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f194e59c700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 0x7f195019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f1947fff700 1 -- 172.21.15.32:0/3830703143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 msgr2=0x7f195019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f1947fff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 0x7f195019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f1947fff700 1 -- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1950105840 msgr2=0x7f19501968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f1947fff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1950105840 0x7f19501968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f1947fff700 1 -- 172.21.15.32:0/3830703143 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380095f0 con 0x7f1950100440 2023-08-11T15:50:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f194e59c700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 0x7f195019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:50:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f1947fff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 0x7f195019d030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f19380098e0 tx=0x7f193800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f19477fe700 1 -- 172.21.15.32:0/3830703143 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193801d070 con 0x7f1950100440 2023-08-11T15:50:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f19477fe700 1 -- 172.21.15.32:0/3830703143 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f193800fb80 con 0x7f1950100440 2023-08-11T15:50:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.257+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1950197640 con 0x7f1950100440 2023-08-11T15:50:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.258+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19501a3a70 con 0x7f1950100440 2023-08-11T15:50:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.258+0000 7f19477fe700 1 -- 172.21.15.32:0/3830703143 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380173f0 con 0x7f1950100440 2023-08-11T15:50:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.261+0000 7f19477fe700 1 -- 172.21.15.32:0/3830703143 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f193800fcf0 con 0x7f1950100440 2023-08-11T15:50:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.262+0000 7f19687a5b80 1 Processor -- start 2023-08-11T15:50:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.262+0000 7f19477fe700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f193c0a00c0 0x7f193c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.262+0000 7f19687a5b80 1 -- start start 2023-08-11T15:50:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f19477fe700 1 -- 172.21.15.32:0/3830703143 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f19380cc4c0 con 0x7f1950100440 2023-08-11T15:50:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f19687a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac8fb5d0 con 0x55b5ac9a6a40 2023-08-11T15:50:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f19687a5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac8f8ef0 con 0x55b5ac9a7400 2023-08-11T15:50:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f19687a5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac8f47e0 con 0x55b5ac9ab110 2023-08-11T15:50:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f194ed9d700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f193c0a00c0 0x7f193c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f194e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f1947fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f194e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:35168/0 (socket says 172.21.15.32:35168) 2023-08-11T15:50:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f194e59c700 1 -- 172.21.15.32:0/3415337109 learned_addr learned my addr 172.21.15.32:0/3415337109 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.263+0000 7f194ed9d700 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f194e59c700 1 -- 172.21.15.32:0/3415337109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 msgr2=0x55b5ac9b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f194e59c700 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f194e59c700 1 -- 172.21.15.32:0/3415337109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a6a40 msgr2=0x55b5ac9a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f194e59c700 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f194e59c700 1 -- 172.21.15.32:0/3415337109 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5ac7745d0 con 0x55b5ac9ab110 2023-08-11T15:50:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f194e59c700 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ab4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1940009dc0 tx=0x7f19400106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa92e469997d2c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f194ed9d700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f193c0a00c0 0x7f193c0a2580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f195019d780 tx=0x7f194800fc00 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f1944ff9700 1 -- 172.21.15.32:0/3415337109 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940010e10 con 0x55b5ac9ab110 2023-08-11T15:50:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f1944ff9700 1 -- 172.21.15.32:0/3415337109 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1940015070 con 0x55b5ac9ab110 2023-08-11T15:50:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.264+0000 7f1944ff9700 1 -- 172.21.15.32:0/3415337109 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400116c0 con 0x55b5ac9ab110 2023-08-11T15:50:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 -- 172.21.15.32:0/3415337109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 msgr2=0x55b5ac9ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ab4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1940009dc0 tx=0x7f19400106b0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 -- 172.21.15.32:0/3415337109 shutdown_connections 2023-08-11T15:50:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ab4f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 --2- 172.21.15.32:0/3415337109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 -- 172.21.15.32:0/3415337109 >> 172.21.15.32:0/3415337109 conn(0x55b5ac8cf8f0 msgr2=0x55b5ac99ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 -- 172.21.15.32:0/3415337109 shutdown_connections 2023-08-11T15:50:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.265+0000 7f19687a5b80 1 -- 172.21.15.32:0/3415337109 wait complete. 2023-08-11T15:50:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.266+0000 7f19687a5b80 1 -- start start 2023-08-11T15:50:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.266+0000 7f19687a5b80 1 Processor -- start 2023-08-11T15:50:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f19687a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac892a60 con 0x55b5ac9a7400 2023-08-11T15:50:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f19687a5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac892760 con 0x55b5ac9a6a40 2023-08-11T15:50:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f19687a5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac7768c0 con 0x55b5ac9ab110 2023-08-11T15:50:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f19687a5b80 10 client.0 _ll_register_callbacks cb 0x55b5ac895a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194ed9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39556/0 (socket says 172.21.15.32:39556) 2023-08-11T15:50:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194e59c700 1 -- 172.21.15.32:0/154884699 learned_addr learned my addr 172.21.15.32:0/154884699 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f1947fff700 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194e59c700 1 -- 172.21.15.32:0/154884699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 msgr2=0x55b5ac9ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194e59c700 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194e59c700 1 -- 172.21.15.32:0/154884699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a7400 msgr2=0x55b5ac9b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.267+0000 7f194e59c700 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f194e59c700 1 -- 172.21.15.32:0/154884699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19400099f0 con 0x55b5ac9a6a40 2023-08-11T15:50:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f194e59c700 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9b2a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1940000c00 tx=0x7f194001c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4b72df7d994c7c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f1935ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:50:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f1935ffb700 1 -- 172.21.15.32:0/154884699 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940020040 con 0x55b5ac9a6a40 2023-08-11T15:50:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f1935ffb700 1 -- 172.21.15.32:0/154884699 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f1940017070 con 0x55b5ac9a6a40 2023-08-11T15:50:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5ac9957e0 con 0x55b5ac9a6a40 2023-08-11T15:50:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f1935ffb700 1 -- 172.21.15.32:0/154884699 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194001cbd0 con 0x55b5ac9a6a40 2023-08-11T15:50:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f19687a5b80 10 client.25721 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.268+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b5ac720d80 con 0x55b5ac9a6a40 2023-08-11T15:50:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f1935ffb700 1 -- 172.21.15.32:0/154884699 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5ac9957e0 con 0x55b5ac9a6a40 2023-08-11T15:50:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f1935ffb700 1 -- 172.21.15.32:0/154884699 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1940011440 con 0x55b5ac9a6a40 2023-08-11T15:50:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f19687a5b80 10 client.25721 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f19687a5b80 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f19687a5b80 20 client.25721 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f19687a5b80 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5ac9c45d0 0x55b5ac9c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f19687a5b80 4 client.25721 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:50:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5ac971480 con 0x55b5ac9c45d0 2023-08-11T15:50:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.269+0000 7f1947fff700 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5ac9c45d0 0x55b5ac9c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.270+0000 7f1947fff700 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5ac9c45d0 0x55b5ac9c69f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d52210ae0c0640 server_cookie=4df91eb804bc5fc4 in_seq=0 out_seq=0 2023-08-11T15:50:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.270+0000 7f1935ffb700 10 client.25721 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.272+0000 7f1935ffb700 1 -- 172.21.15.32:0/154884699 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55b5ac971480 con 0x55b5ac9c45d0 2023-08-11T15:50:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.272+0000 7f1935ffb700 10 client.25721 handle_command_reply: tid=0 2023-08-11T15:50:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.272+0000 7f19687a5b80 1 client.25721 shutdown 2023-08-11T15:50:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 msgr2=0x55b5ac9b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9b2a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1940000c00 tx=0x7f194001c6b0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 shutdown_connections 2023-08-11T15:50:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5ac9c45d0 0x55b5ac9c69f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9ab110 0x55b5ac9ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f1935ffb700 0 client.25721 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9b2a60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 --2- 172.21.15.32:0/154884699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9a7400 0x55b5ac9b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 >> 172.21.15.32:0/154884699 conn(0x55b5ac8cf8f0 msgr2=0x55b5ac99c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 shutdown_connections 2023-08-11T15:50:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 1 -- 172.21.15.32:0/154884699 wait complete. 2023-08-11T15:50:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.273+0000 7f19687a5b80 20 client.25721 trim_cache size 0 max 16384 2023-08-11T15:50:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.289+0000 7f19687a5b80 1 Processor -- start 2023-08-11T15:50:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f19687a5b80 1 -- start start 2023-08-11T15:50:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a7400 0x55b5ac9a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5ac9a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f19687a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac9a6ea0 con 0x55b5ac9ab110 2023-08-11T15:50:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f19687a5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac9a7260 con 0x55b5ac9a6a40 2023-08-11T15:50:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f19687a5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ac9955a0 con 0x55b5ac9a7400 2023-08-11T15:50:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f194e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f194e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39570/0 (socket says 172.21.15.32:39570) 2023-08-11T15:50:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f194e59c700 1 -- 172.21.15.32:0/2914873637 learned_addr learned my addr 172.21.15.32:0/2914873637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f1947fff700 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a7400 0x55b5ac9a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.290+0000 7f194ed9d700 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5ac9a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f194e59c700 1 -- 172.21.15.32:0/2914873637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a7400 msgr2=0x55b5ac9a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f194e59c700 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a7400 0x55b5ac9a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f194e59c700 1 -- 172.21.15.32:0/2914873637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 msgr2=0x55b5ac9a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f194e59c700 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5ac9a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f194e59c700 1 -- 172.21.15.32:0/2914873637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5ac709c00 con 0x55b5ac9a6a40 2023-08-11T15:50:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f194e59c700 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9c69c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1940027040 tx=0x7f19400095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec389db8c116c545 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f19457fa700 1 -- 172.21.15.32:0/2914873637 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940020070 con 0x55b5ac9a6a40 2023-08-11T15:50:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f19457fa700 1 -- 172.21.15.32:0/2914873637 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1940009810 con 0x55b5ac9a6a40 2023-08-11T15:50:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.291+0000 7f19457fa700 1 -- 172.21.15.32:0/2914873637 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400052f0 con 0x55b5ac9a6a40 2023-08-11T15:50:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 -- 172.21.15.32:0/2914873637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 msgr2=0x55b5ac9c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9c69c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1940027040 tx=0x7f19400095c0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 -- 172.21.15.32:0/2914873637 shutdown_connections 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5ac9a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9c69c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 --2- 172.21.15.32:0/2914873637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a7400 0x55b5ac9a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 -- 172.21.15.32:0/2914873637 >> 172.21.15.32:0/2914873637 conn(0x55b5ac8cf8f0 msgr2=0x55b5ac9aa500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 -- 172.21.15.32:0/2914873637 shutdown_connections 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 -- 172.21.15.32:0/2914873637 wait complete. 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.292+0000 7f19687a5b80 1 -- start start 2023-08-11T15:50:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 1 Processor -- start 2023-08-11T15:50:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9bd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5aca17290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5aca177d0 con 0x55b5ac9ab110 2023-08-11T15:50:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5aca17910 con 0x55b5ac9a7400 2023-08-11T15:50:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5aca17a80 con 0x55b5ac9a6a40 2023-08-11T15:50:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f19687a5b80 10 client.0 _ll_register_callbacks cb 0x55b5ac895a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f1947fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9bd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f194e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f194ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5aca17290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f1947fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9bd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39586/0 (socket says 172.21.15.32:39586) 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f194e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9bd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:35204/0 (socket says 172.21.15.32:35204) 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.293+0000 7f1947fff700 1 -- 172.21.15.32:0/760114887 learned_addr learned my addr 172.21.15.32:0/760114887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1947fff700 1 -- 172.21.15.32:0/760114887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a6a40 msgr2=0x55b5ac9bd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1947fff700 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9bd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1947fff700 1 -- 172.21.15.32:0/760114887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 msgr2=0x55b5aca17290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1947fff700 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5aca17290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1947fff700 1 -- 172.21.15.32:0/760114887 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5ac720d80 con 0x55b5ac9a7400 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1947fff700 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9bd9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1938095e50 tx=0x7f193801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46af0575fe644536 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1935ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1935ffb700 1 -- 172.21.15.32:0/760114887 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380cc330 con 0x55b5ac9a7400 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f1935ffb700 1 -- 172.21.15.32:0/760114887 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f193809d780 con 0x55b5ac9a7400 2023-08-11T15:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.294+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1940015070 con 0x55b5ac9a7400 2023-08-11T15:50:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f1935ffb700 1 -- 172.21.15.32:0/760114887 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19380a16d0 con 0x55b5ac9a7400 2023-08-11T15:50:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 10 client.25727 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b5ac7471f0 con 0x55b5ac9a7400 2023-08-11T15:50:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f1935ffb700 1 -- 172.21.15.32:0/760114887 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1940015070 con 0x55b5ac9a7400 2023-08-11T15:50:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f1935ffb700 1 -- 172.21.15.32:0/760114887 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f19380a1240 con 0x55b5ac9a7400 2023-08-11T15:50:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 10 client.25727 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 20 client.25727 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5aca18e60 0x55b5aca1b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 4 client.25727 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:50:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5ac8ed7c0 con 0x55b5aca18e60 2023-08-11T15:50:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.295+0000 7f194e59c700 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5aca18e60 0x55b5aca1b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.296+0000 7f194e59c700 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5aca18e60 0x55b5aca1b280 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bc1157533e85ad server_cookie=eab0be23c866e3a1 in_seq=0 out_seq=0 2023-08-11T15:50:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.296+0000 7f1935ffb700 10 client.25727 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.538+0000 7f1935ffb700 1 -- 172.21.15.32:0/760114887 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5ac8ed7c0 con 0x55b5aca18e60 2023-08-11T15:50:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.538+0000 7f1935ffb700 10 client.25727 handle_command_reply: tid=0 2023-08-11T15:50:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.538+0000 7f19687a5b80 1 client.25727 shutdown 2023-08-11T15:50:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.538+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 msgr2=0x55b5ac9bd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.538+0000 7f19687a5b80 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9bd9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1938095e50 tx=0x7f193801a040 comp rx=0 tx=0).stop 2023-08-11T15:50:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.540+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 shutdown_connections 2023-08-11T15:50:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.540+0000 7f19687a5b80 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55b5aca18e60 0x55b5aca1b280 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.540+0000 7f19687a5b80 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5ac9ab110 0x55b5aca17290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.540+0000 7f19687a5b80 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5ac9a6a40 0x55b5ac9bd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.540+0000 7f19687a5b80 1 --2- 172.21.15.32:0/760114887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5ac9a7400 0x55b5ac9bd9e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.540+0000 7f1935ffb700 0 client.25727 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.540+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 >> 172.21.15.32:0/760114887 conn(0x55b5ac8cf8f0 msgr2=0x55b5ac9af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.541+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 shutdown_connections 2023-08-11T15:50:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.541+0000 7f19687a5b80 1 -- 172.21.15.32:0/760114887 wait complete. 2023-08-11T15:50:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.541+0000 7f19687a5b80 20 client.25727 trim_cache size 0 max 16384 2023-08-11T15:50:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:50:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:50:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:50:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:50:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f193c0a00c0 msgr2=0x7f193c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f193c0a00c0 0x7f193c0a2580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f195019d780 tx=0x7f194800fc00 comp rx=0 tx=0).stop 2023-08-11T15:50:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 msgr2=0x7f195019d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 0x7f195019d030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f19380098e0 tx=0x7f193800f690 comp rx=0 tx=0).stop 2023-08-11T15:50:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 shutdown_connections 2023-08-11T15:50:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f193c0a00c0 0x7f193c0a2580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1950105840 0x7f19501968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19500ffa80 0x7f195019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 --2- 172.21.15.32:0/3830703143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1950100440 0x7f195019d030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 >> 172.21.15.32:0/3830703143 conn(0x7f1950075250 msgr2=0x7f1950067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 shutdown_connections 2023-08-11T15:50:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:45.542+0000 7f1954dff700 1 -- 172.21.15.32:0/3830703143 wait complete. 2023-08-11T15:50:45.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 438404 12.51 MB/sec execute 61 sec latency 191.297 ms 2023-08-11T15:50:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:46 smithi032 ceph-mon[97103]: pgmap v128: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 269 op/s 2023-08-11T15:50:46.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:46 smithi192 ceph-mon[130806]: pgmap v128: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 269 op/s 2023-08-11T15:50:46.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:46 smithi134 ceph-mon[130447]: pgmap v128: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 269 op/s 2023-08-11T15:50:46.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 440739 12.54 MB/sec execute 62 sec latency 7.934 ms 2023-08-11T15:50:47.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 443771 12.63 MB/sec execute 63 sec latency 7.019 ms 2023-08-11T15:50:48.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:47 smithi134 ceph-mon[130447]: pgmap v129: 129 pgs: 129 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 436 op/s 2023-08-11T15:50:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:47 smithi032 ceph-mon[97103]: pgmap v129: 129 pgs: 129 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 436 op/s 2023-08-11T15:50:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:47 smithi192 ceph-mon[130806]: pgmap v129: 129 pgs: 129 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 436 op/s 2023-08-11T15:50:48.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 446413 12.63 MB/sec execute 64 sec latency 20.120 ms 2023-08-11T15:50:49.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 448641 12.62 MB/sec execute 65 sec latency 23.608 ms 2023-08-11T15:50:50.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:50 smithi032 ceph-mon[97103]: pgmap v130: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 328 op/s 2023-08-11T15:50:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:50 smithi192 ceph-mon[130806]: pgmap v130: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 328 op/s 2023-08-11T15:50:50.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:50 smithi134 ceph-mon[130447]: pgmap v130: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 328 op/s 2023-08-11T15:50:50.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 451051 12.64 MB/sec execute 66 sec latency 85.993 ms 2023-08-11T15:50:51.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 454100 12.71 MB/sec execute 67 sec latency 11.859 ms 2023-08-11T15:50:52.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:52 smithi032 ceph-mon[97103]: pgmap v131: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 293 op/s 2023-08-11T15:50:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:52 smithi192 ceph-mon[130806]: pgmap v131: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 293 op/s 2023-08-11T15:50:52.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:52 smithi134 ceph-mon[130447]: pgmap v131: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 293 op/s 2023-08-11T15:50:52.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 456179 12.63 MB/sec execute 68 sec latency 24.222 ms 2023-08-11T15:50:53.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 458308 12.65 MB/sec execute 69 sec latency 22.564 ms 2023-08-11T15:50:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:54 smithi032 ceph-mon[97103]: pgmap v132: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 373 op/s 2023-08-11T15:50:54.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:54 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:50:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:54 smithi192 ceph-mon[130806]: pgmap v132: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 373 op/s 2023-08-11T15:50:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:54 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:50:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:54 smithi134 ceph-mon[130447]: pgmap v132: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 373 op/s 2023-08-11T15:50:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:54 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:50:54.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 460482 12.85 MB/sec execute 70 sec latency 47.014 ms 2023-08-11T15:50:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:55 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:50:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:55 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:50:55.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:55 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:50:55.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:55 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:50:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:55 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:50:55.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:55 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:50:55.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:55 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:50:55.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:55 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:50:55.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:55 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:50:55.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 462777 12.86 MB/sec execute 71 sec latency 40.379 ms 2023-08-11T15:50:56.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:56 smithi032 ceph-mon[97103]: pgmap v133: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 291 op/s 2023-08-11T15:50:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:56 smithi192 ceph-mon[130806]: pgmap v133: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 291 op/s 2023-08-11T15:50:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:56 smithi134 ceph-mon[130447]: pgmap v133: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 291 op/s 2023-08-11T15:50:56.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 465880 12.94 MB/sec execute 72 sec latency 6.397 ms 2023-08-11T15:50:57.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 468960 13.00 MB/sec execute 73 sec latency 6.582 ms 2023-08-11T15:50:58.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:50:57 smithi134 ceph-mon[130447]: pgmap v134: 129 pgs: 129 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2023-08-11T15:50:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:50:57 smithi032 ceph-mon[97103]: pgmap v134: 129 pgs: 129 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2023-08-11T15:50:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:50:57 smithi192 ceph-mon[130806]: pgmap v134: 129 pgs: 129 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2023-08-11T15:50:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.701+0000 7f860efff700 1 -- 172.21.15.32:0/4196066027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608072d50 msgr2=0x7f8608073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.701+0000 7f860efff700 1 --2- 172.21.15.32:0/4196066027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608072d50 0x7f8608073130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009990 tx=0x7f85fc009ca0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.701+0000 7f860efff700 1 -- 172.21.15.32:0/4196066027 shutdown_connections 2023-08-11T15:50:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.701+0000 7f860efff700 1 --2- 172.21.15.32:0/4196066027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860810d0e0 0x7f860810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.701+0000 7f860efff700 1 --2- 172.21.15.32:0/4196066027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608072d50 0x7f8608073130 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.701+0000 7f860efff700 1 --2- 172.21.15.32:0/4196066027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8608073670 0x7f860810cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.701+0000 7f860efff700 1 -- 172.21.15.32:0/4196066027 >> 172.21.15.32:0/4196066027 conn(0x7f86080fc370 msgr2=0x7f86080fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.702+0000 7f860efff700 1 -- 172.21.15.32:0/4196066027 shutdown_connections 2023-08-11T15:50:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.702+0000 7f860efff700 1 -- 172.21.15.32:0/4196066027 wait complete. 2023-08-11T15:50:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.703+0000 7f860efff700 1 Processor -- start 2023-08-11T15:50:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.703+0000 7f860efff700 1 -- start start 2023-08-11T15:50:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.703+0000 7f860efff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608072d50 0x7f860819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.703+0000 7f860efff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608073670 0x7f860819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.703+0000 7f860efff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f860810d0e0 0x7f8608198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860efff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081990b0 con 0x7f8608072d50 2023-08-11T15:50:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860efff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608199220 con 0x7f860810d0e0 2023-08-11T15:50:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860efff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081993e0 con 0x7f8608073670 2023-08-11T15:50:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f860810d0e0 0x7f8608198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608072d50 0x7f860819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f860810d0e0 0x7f8608198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44220/0 (socket says 172.21.15.32:44220) 2023-08-11T15:50:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860d59c700 1 -- 172.21.15.32:0/4185205461 learned_addr learned my addr 172.21.15.32:0/4185205461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f8607fff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608073670 0x7f860819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f8607fff700 1 -- 172.21.15.32:0/4185205461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f860810d0e0 msgr2=0x7f8608198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f8607fff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f860810d0e0 0x7f8608198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f8607fff700 1 -- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608072d50 msgr2=0x7f860819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f8607fff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608072d50 0x7f860819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f8607fff700 1 -- 172.21.15.32:0/4185205461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0095f0 con 0x7f8608073670 2023-08-11T15:50:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860d59c700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f860810d0e0 0x7f8608198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.704+0000 7f860cd9b700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608072d50 0x7f860819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.705+0000 7f8607fff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608073670 0x7f860819f120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85f400e9f0 tx=0x7f85f400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.705+0000 7f8605ffb700 1 -- 172.21.15.32:0/4185205461 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400c9c0 con 0x7f8608073670 2023-08-11T15:50:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.705+0000 7f8605ffb700 1 -- 172.21.15.32:0/4185205461 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85f4010430 con 0x7f8608073670 2023-08-11T15:50:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.705+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86081996c0 con 0x7f8608073670 2023-08-11T15:50:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.705+0000 7f8605ffb700 1 -- 172.21.15.32:0/4185205461 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4018680 con 0x7f8608073670 2023-08-11T15:50:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.705+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86081a5bd0 con 0x7f8608073670 2023-08-11T15:50:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.709+0000 7f8605ffb700 1 -- 172.21.15.32:0/4185205461 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f85f4010860 con 0x7f8608073670 2023-08-11T15:50:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.711+0000 7f862281fb80 1 Processor -- start 2023-08-11T15:50:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.711+0000 7f862281fb80 1 -- start start 2023-08-11T15:50:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f862281fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb1c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f862281fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f862281fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb20b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f862281fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93ba6c5d0 con 0x55a93bb20720 2023-08-11T15:50:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f862281fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93ba69ef0 con 0x55a93bb1c030 2023-08-11T15:50:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f862281fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93ba657e0 con 0x55a93bb1c950 2023-08-11T15:50:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f8605ffb700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f85f80a00c0 0x7f85f80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f8605ffb700 1 -- 172.21.15.32:0/4185205461 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f85f4014070 con 0x7f8608073670 2023-08-11T15:50:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb201e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:46998/0 (socket says 172.21.15.32:46998) 2023-08-11T15:50:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f860cd9b700 1 -- 172.21.15.32:0/600068045 learned_addr learned my addr 172.21.15.32:0/600068045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f860d59c700 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb1c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.712+0000 7f8607fff700 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb20b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.713+0000 7f860d59c700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f85f80a00c0 0x7f85f80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.713+0000 7f860d59c700 1 -- 172.21.15.32:0/600068045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 msgr2=0x55a93bb201e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.713+0000 7f860d59c700 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb201e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.713+0000 7f860d59c700 1 -- 172.21.15.32:0/600068045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 msgr2=0x55a93bb20b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.713+0000 7f860d59c700 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb20b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.713+0000 7f860d59c700 1 -- 172.21.15.32:0/600068045 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a93b8e55d0 con 0x55a93bb1c030 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.714+0000 7f860d59c700 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb1c410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f860819f8e0 tx=0x7f860000efc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc922cd10f339bcd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.714+0000 7f860d59c700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f85f80a00c0 0x7f85f80a2580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8600009000 tx=0x7f860001c000 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.714+0000 7f85f2ffd700 1 -- 172.21.15.32:0/600068045 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600007700 con 0x55a93bb1c030 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.714+0000 7f85f2ffd700 1 -- 172.21.15.32:0/600068045 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f860000f040 con 0x55a93bb1c030 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.714+0000 7f85f2ffd700 1 -- 172.21.15.32:0/600068045 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600021970 con 0x55a93bb1c030 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 -- 172.21.15.32:0/600068045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 msgr2=0x55a93bb1c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb1c410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f860819f8e0 tx=0x7f860000efc0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 -- 172.21.15.32:0/600068045 shutdown_connections 2023-08-11T15:50:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb20b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb1c410 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 --2- 172.21.15.32:0/600068045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 -- 172.21.15.32:0/600068045 >> 172.21.15.32:0/600068045 conn(0x55a93ba408f0 msgr2=0x55a93bb0de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 -- 172.21.15.32:0/600068045 shutdown_connections 2023-08-11T15:50:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.715+0000 7f862281fb80 1 -- 172.21.15.32:0/600068045 wait complete. 2023-08-11T15:50:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.716+0000 7f862281fb80 1 -- start start 2023-08-11T15:50:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.716+0000 7f862281fb80 1 Processor -- start 2023-08-11T15:50:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f862281fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb27e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f862281fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f862281fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f862281fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93ba03a60 con 0x55a93bb1c950 2023-08-11T15:50:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f862281fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93ba03760 con 0x55a93bb20720 2023-08-11T15:50:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f862281fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93b8e78c0 con 0x55a93bb1c030 2023-08-11T15:50:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f862281fb80 10 client.0 _ll_register_callbacks cb 0x55a93ba06a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb27e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f8607fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb27e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:47006/0 (socket says 172.21.15.32:47006) 2023-08-11T15:50:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860cd9b700 1 -- 172.21.15.32:0/3657469783 learned_addr learned my addr 172.21.15.32:0/3657469783 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860d59c700 1 -- 172.21.15.32:0/3657469783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 msgr2=0x55a93bb27e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860d59c700 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb27e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860d59c700 1 -- 172.21.15.32:0/3657469783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 msgr2=0x55a93bb25120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860d59c700 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb25120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.717+0000 7f860d59c700 1 -- 172.21.15.32:0/3657469783 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860000eba0 con 0x55a93bb20720 2023-08-11T15:50:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f860d59c700 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb256e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8600009960 tx=0x7f8600021430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb47a3d44b992ec server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f85e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:50:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f85e7fff700 1 -- 172.21.15.32:0/3657469783 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600021a20 con 0x55a93bb20720 2023-08-11T15:50:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a93bb067e0 con 0x55a93bb20720 2023-08-11T15:50:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f85e7fff700 1 -- 172.21.15.32:0/3657469783 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f8600021b80 con 0x55a93bb20720 2023-08-11T15:50:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f85e7fff700 1 -- 172.21.15.32:0/3657469783 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8600007d50 con 0x55a93bb20720 2023-08-11T15:50:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f862281fb80 10 client.25739 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a93b891d80 con 0x55a93bb20720 2023-08-11T15:50:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.718+0000 7f85e7fff700 1 -- 172.21.15.32:0/3657469783 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a93bb067e0 con 0x55a93bb20720 2023-08-11T15:50:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f85e7fff700 1 -- 172.21.15.32:0/3657469783 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f860002abd0 con 0x55a93bb20720 2023-08-11T15:50:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f862281fb80 10 client.25739 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f862281fb80 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f862281fb80 20 client.25739 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f862281fb80 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb35730 0x55a93bb37b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f862281fb80 4 client.25739 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a93bae2480 con 0x55a93bb35730 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f860cd9b700 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb35730 0x55a93bb37b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f85e7fff700 10 client.25739 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.719+0000 7f860cd9b700 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb35730 0x55a93bb37b50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4920759c3f9224b3 server_cookie=b03910f724300e59 in_seq=0 out_seq=0 2023-08-11T15:50:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f85e7fff700 1 -- 172.21.15.32:0/3657469783 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a93bae2480 con 0x55a93bb35730 2023-08-11T15:50:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f85e7fff700 10 client.25739 handle_command_reply: tid=0 2023-08-11T15:50:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 client.25739 shutdown 2023-08-11T15:50:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 msgr2=0x55a93bb256e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb256e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8600009960 tx=0x7f8600021430 comp rx=0 tx=0).stop 2023-08-11T15:50:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 shutdown_connections 2023-08-11T15:50:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb35730 0x55a93bb37b50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb256e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb27e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 --2- 172.21.15.32:0/3657469783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f85e7fff700 0 client.25739 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.720+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 >> 172.21.15.32:0/3657469783 conn(0x55a93ba408f0 msgr2=0x55a93bb0de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.721+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 shutdown_connections 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.721+0000 7f862281fb80 1 -- 172.21.15.32:0/3657469783 wait complete. 2023-08-11T15:50:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.721+0000 7f862281fb80 20 client.25739 trim_cache size 0 max 16384 2023-08-11T15:50:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 Processor -- start 2023-08-11T15:50:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 -- start start 2023-08-11T15:50:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb27930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93bae24b0 con 0x55a93bb20720 2023-08-11T15:50:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93bb317c0 con 0x55a93bb1c030 2023-08-11T15:50:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.736+0000 7f862281fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93bb067c0 con 0x55a93bb1c950 2023-08-11T15:50:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f8607fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb2c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:44250/0 (socket says 172.21.15.32:44250) 2023-08-11T15:50:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 -- 172.21.15.32:0/1674805259 learned_addr learned my addr 172.21.15.32:0/1674805259 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860d59c700 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb27930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 -- 172.21.15.32:0/1674805259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 msgr2=0x55a93bb319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 -- 172.21.15.32:0/1674805259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 msgr2=0x55a93bb27930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb27930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 -- 172.21.15.32:0/1674805259 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a93b87ac00 con 0x55a93bb1c030 2023-08-11T15:50:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.737+0000 7f860cd9b700 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb2c840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00b6f0 tx=0x7f85fc005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102bb9ea8cc70f86 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f85f37fe700 1 -- 172.21.15.32:0/1674805259 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc01d040 con 0x55a93bb1c030 2023-08-11T15:50:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f85f37fe700 1 -- 172.21.15.32:0/1674805259 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85fc01e040 con 0x55a93bb1c030 2023-08-11T15:50:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f85f37fe700 1 -- 172.21.15.32:0/1674805259 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc005520 con 0x55a93bb1c030 2023-08-11T15:50:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 -- 172.21.15.32:0/1674805259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 msgr2=0x55a93bb2c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb2c840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00b6f0 tx=0x7f85fc005fb0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 -- 172.21.15.32:0/1674805259 shutdown_connections 2023-08-11T15:50:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb20720 0x55a93bb27930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb1c030 0x55a93bb2c840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 --2- 172.21.15.32:0/1674805259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c950 0x55a93bb319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 -- 172.21.15.32:0/1674805259 >> 172.21.15.32:0/1674805259 conn(0x55a93ba408f0 msgr2=0x55a93bb0dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 -- 172.21.15.32:0/1674805259 shutdown_connections 2023-08-11T15:50:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.738+0000 7f862281fb80 1 -- 172.21.15.32:0/1674805259 wait complete. 2023-08-11T15:50:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.739+0000 7f862281fb80 1 -- start start 2023-08-11T15:50:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.739+0000 7f862281fb80 1 Processor -- start 2023-08-11T15:50:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f862281fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f862281fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb2c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f862281fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f862281fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93bb2ae10 con 0x55a93bb1c950 2023-08-11T15:50:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f862281fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93bb2af80 con 0x55a93bb20720 2023-08-11T15:50:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f862281fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93bb2b140 con 0x55a93bb1c030 2023-08-11T15:50:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f862281fb80 10 client.0 _ll_register_callbacks cb 0x7f85f40951c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:50:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f860cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb2c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:47030/0 (socket says 172.21.15.32:47030) 2023-08-11T15:50:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f860cd9b700 1 -- 172.21.15.32:0/1180190033 learned_addr learned my addr 172.21.15.32:0/1180190033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:50:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f8607fff700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb2c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.740+0000 7f860d59c700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f860cd9b700 1 -- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 msgr2=0x55a93bb2a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f860cd9b700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb2a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f860cd9b700 1 -- 172.21.15.32:0/1180190033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 msgr2=0x55a93bb2c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:50:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f860cd9b700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb2c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f860cd9b700 1 -- 172.21.15.32:0/1180190033 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8608110210 con 0x55a93bb1c030 2023-08-11T15:50:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f8607fff700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb2c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f860d59c700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb2a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:50:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f860cd9b700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb2c030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009ce0 tx=0x7f85fc029180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211d0a90eecda488 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:50:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f85e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:50:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f85e7fff700 1 -- 172.21.15.32:0/1180190033 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc01d070 con 0x55a93bb1c030 2023-08-11T15:50:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f860000f040 con 0x55a93bb1c030 2023-08-11T15:50:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.741+0000 7f85e7fff700 1 -- 172.21.15.32:0/1180190033 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f85fc029c30 con 0x55a93bb1c030 2023-08-11T15:50:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f85e7fff700 1 -- 172.21.15.32:0/1180190033 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85fc029df0 con 0x55a93bb1c030 2023-08-11T15:50:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 10 client.25747 fetch_fsmap learned FSMap version 27 2023-08-11T15:50:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a93b8b81f0 con 0x55a93bb1c030 2023-08-11T15:50:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f85e7fff700 1 -- 172.21.15.32:0/1180190033 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f860000f040 con 0x55a93bb1c030 2023-08-11T15:50:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f85e7fff700 1 -- 172.21.15.32:0/1180190033 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f85fc0326f0 con 0x55a93bb1c030 2023-08-11T15:50:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 10 client.25747 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:50:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:50:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 20 client.25747 populate_metadata read hostname 'smithi032' 2023-08-11T15:50:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb8a2f0 0x55a93bb8c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:50:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 4 client.25747 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:50:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.742+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a93ba5e7c0 con 0x55a93bb8a2f0 2023-08-11T15:50:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.743+0000 7f8607fff700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb8a2f0 0x55a93bb8c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:50:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.743+0000 7f8607fff700 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb8a2f0 0x55a93bb8c6e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d89ad86dd999428 server_cookie=5cb50d8b72806d in_seq=0 out_seq=0 2023-08-11T15:50:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:58.743+0000 7f85e7fff700 10 client.25747 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:58.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 471197 12.93 MB/sec execute 74 sec latency 28.404 ms 2023-08-11T15:50:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f85e7fff700 1 -- 172.21.15.32:0/1180190033 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a93ba5e7c0 con 0x55a93bb8a2f0 2023-08-11T15:50:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f85e7fff700 10 client.25747 handle_command_reply: tid=0 2023-08-11T15:50:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 client.25747 shutdown 2023-08-11T15:50:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 msgr2=0x55a93bb2c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb2c030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009ce0 tx=0x7f85fc029180 comp rx=0 tx=0).stop 2023-08-11T15:50:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 shutdown_connections 2023-08-11T15:50:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a93bb8a2f0 0x55a93bb8c6e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a93bb20720 0x55a93bb2a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a93bb1c030 0x55a93bb2c030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 --2- 172.21.15.32:0/1180190033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a93bb1c950 0x55a93bb2c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 >> 172.21.15.32:0/1180190033 conn(0x55a93ba408f0 msgr2=0x55a93bb250c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.254+0000 7f85e7fff700 0 client.25747 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:50:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.255+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 shutdown_connections 2023-08-11T15:50:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.255+0000 7f862281fb80 1 -- 172.21.15.32:0/1180190033 wait complete. 2023-08-11T15:50:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.256+0000 7f862281fb80 20 client.25747 trim_cache size 0 max 16384 2023-08-11T15:50:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:50:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:50:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:50:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:50:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.257+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f85f80a00c0 msgr2=0x7f85f80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.257+0000 7f860efff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f85f80a00c0 0x7f85f80a2580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8600009000 tx=0x7f860001c000 comp rx=0 tx=0).stop 2023-08-11T15:50:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.257+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608073670 msgr2=0x7f860819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:50:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.257+0000 7f860efff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608073670 0x7f860819f120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85f400e9f0 tx=0x7f85f400ed00 comp rx=0 tx=0).stop 2023-08-11T15:50:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 shutdown_connections 2023-08-11T15:50:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f85f80a00c0 0x7f85f80a2580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8608072d50 0x7f860819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f860810d0e0 0x7f8608198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 --2- 172.21.15.32:0/4185205461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8608073670 0x7f860819f120 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:50:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 >> 172.21.15.32:0/4185205461 conn(0x7f86080fc370 msgr2=0x7f860810f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:50:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 shutdown_connections 2023-08-11T15:50:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:50:59.258+0000 7f860efff700 1 -- 172.21.15.32:0/4185205461 wait complete. 2023-08-11T15:50:59.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 472811 12.93 MB/sec execute 75 sec latency 199.569 ms 2023-08-11T15:51:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:00 smithi032 ceph-mon[97103]: pgmap v135: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 210 op/s 2023-08-11T15:51:00.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:00 smithi192 ceph-mon[130806]: pgmap v135: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 210 op/s 2023-08-11T15:51:00.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:00 smithi134 ceph-mon[130447]: pgmap v135: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 210 op/s 2023-08-11T15:51:00.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 473426 12.77 MB/sec execute 76 sec latency 297.258 ms 2023-08-11T15:51:01.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 473574 12.62 MB/sec execute 77 sec latency 247.770 ms 2023-08-11T15:51:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:02 smithi032 ceph-mon[97103]: pgmap v136: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2023-08-11T15:51:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:02 smithi192 ceph-mon[130806]: pgmap v136: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2023-08-11T15:51:02.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:02 smithi134 ceph-mon[130447]: pgmap v136: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2023-08-11T15:51:02.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 476395 12.68 MB/sec execute 78 sec latency 53.073 ms 2023-08-11T15:51:03.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 479295 12.69 MB/sec execute 79 sec latency 6.943 ms 2023-08-11T15:51:04.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:04 smithi192 ceph-mon[130806]: pgmap v137: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 573 op/s 2023-08-11T15:51:04.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:04 smithi134 ceph-mon[130447]: pgmap v137: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 573 op/s 2023-08-11T15:51:04.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 481562 12.67 MB/sec execute 80 sec latency 11.172 ms 2023-08-11T15:51:04.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:04 smithi032 ceph-mon[97103]: pgmap v137: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 573 op/s 2023-08-11T15:51:05.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 483278 12.67 MB/sec execute 81 sec latency 203.946 ms 2023-08-11T15:51:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:06 smithi192 ceph-mon[130806]: pgmap v138: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 4.3 MiB/s wr, 492 op/s 2023-08-11T15:51:06.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:06 smithi134 ceph-mon[130447]: pgmap v138: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 4.3 MiB/s wr, 492 op/s 2023-08-11T15:51:06.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 484397 12.56 MB/sec execute 82 sec latency 30.680 ms 2023-08-11T15:51:06.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:06 smithi032 ceph-mon[97103]: pgmap v138: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 4.3 MiB/s wr, 492 op/s 2023-08-11T15:51:07.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 487244 12.61 MB/sec execute 83 sec latency 17.978 ms 2023-08-11T15:51:08.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:07 smithi134 ceph-mon[130447]: pgmap v139: 129 pgs: 129 active+clean; 57 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.8 MiB/s wr, 633 op/s 2023-08-11T15:51:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:07 smithi032 ceph-mon[97103]: pgmap v139: 129 pgs: 129 active+clean; 57 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.8 MiB/s wr, 633 op/s 2023-08-11T15:51:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:07 smithi192 ceph-mon[130806]: pgmap v139: 129 pgs: 129 active+clean; 57 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.8 MiB/s wr, 633 op/s 2023-08-11T15:51:08.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 490314 12.67 MB/sec execute 84 sec latency 7.254 ms 2023-08-11T15:51:09.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 492182 12.60 MB/sec execute 85 sec latency 46.881 ms 2023-08-11T15:51:10.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:10 smithi032 ceph-mon[97103]: pgmap v140: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.9 MiB/s wr, 590 op/s 2023-08-11T15:51:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:10 smithi192 ceph-mon[130806]: pgmap v140: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.9 MiB/s wr, 590 op/s 2023-08-11T15:51:10.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:10 smithi134 ceph-mon[130447]: pgmap v140: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.9 MiB/s wr, 590 op/s 2023-08-11T15:51:10.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 494530 12.62 MB/sec execute 86 sec latency 56.313 ms 2023-08-11T15:51:11.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 497534 12.67 MB/sec execute 87 sec latency 7.725 ms 2023-08-11T15:51:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 -- 172.21.15.32:0/1404337115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 msgr2=0x7f9408106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 --2- 172.21.15.32:0/1404337115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f9408106cb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9404009990 tx=0x7f9404009ca0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 -- 172.21.15.32:0/1404337115 shutdown_connections 2023-08-11T15:51:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 --2- 172.21.15.32:0/1404337115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94080686d0 0x7f940810d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 --2- 172.21.15.32:0/1404337115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f9408106cb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 --2- 172.21.15.32:0/1404337115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9408067d10 0x7f9408068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 -- 172.21.15.32:0/1404337115 >> 172.21.15.32:0/1404337115 conn(0x7f9408075250 msgr2=0x7f9408075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 -- 172.21.15.32:0/1404337115 shutdown_connections 2023-08-11T15:51:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.432+0000 7f940f9ff700 1 -- 172.21.15.32:0/1404337115 wait complete. 2023-08-11T15:51:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.434+0000 7f940f9ff700 1 Processor -- start 2023-08-11T15:51:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.434+0000 7f940f9ff700 1 -- start start 2023-08-11T15:51:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940f9ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9408067d10 0x7f940819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940f9ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94080686d0 0x7f940819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940f9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f94081968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940f9ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408197090 con 0x7f9408067d10 2023-08-11T15:51:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940f9ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408197200 con 0x7f94080686d0 2023-08-11T15:51:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940f9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94081973c0 con 0x7f94081068d0 2023-08-11T15:51:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940df9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f94081968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94080686d0 0x7f940819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940df9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f94081968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:53380/0 (socket says 172.21.15.32:53380) 2023-08-11T15:51:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.435+0000 7f940df9c700 1 -- 172.21.15.32:0/3209227725 learned_addr learned my addr 172.21.15.32:0/3209227725 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940d79b700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9408067d10 0x7f940819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940df9c700 1 -- 172.21.15.32:0/3209227725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94080686d0 msgr2=0x7f940819d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940df9c700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94080686d0 0x7f940819d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940df9c700 1 -- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9408067d10 msgr2=0x7f940819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940df9c700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9408067d10 0x7f940819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940df9c700 1 -- 172.21.15.32:0/3209227725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94040095f0 con 0x7f94081068d0 2023-08-11T15:51:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940df9c700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f94081968c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9400007ee0 tx=0x7f940000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940cf9a700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94080686d0 0x7f940819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f93fe7fc700 1 -- 172.21.15.32:0/3209227725 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400017070 con 0x7f94081068d0 2023-08-11T15:51:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f93fe7fc700 1 -- 172.21.15.32:0/3209227725 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f940000f040 con 0x7f94081068d0 2023-08-11T15:51:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9408197640 con 0x7f94081068d0 2023-08-11T15:51:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.436+0000 7f93fe7fc700 1 -- 172.21.15.32:0/3209227725 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400005740 con 0x7f94081068d0 2023-08-11T15:51:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.437+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94080ffc60 con 0x7f94081068d0 2023-08-11T15:51:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.441+0000 7f93fe7fc700 1 -- 172.21.15.32:0/3209227725 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f94000058a0 con 0x7f94081068d0 2023-08-11T15:51:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.443+0000 7f94232e4b80 1 Processor -- start 2023-08-11T15:51:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.444+0000 7f93fe7fc700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f93f40a01c0 0x7f93f40a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.444+0000 7f93fe7fc700 1 -- 172.21.15.32:0/3209227725 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f94000cb070 con 0x7f94081068d0 2023-08-11T15:51:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.444+0000 7f940cf9a700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f93f40a01c0 0x7f93f40a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.444+0000 7f94232e4b80 1 -- start start 2023-08-11T15:51:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a15690 0x5602a4a15a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a10800 0x5602a4a1c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f94232e4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a49605d0 con 0x5602a4a15690 2023-08-11T15:51:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f94232e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a495def0 con 0x5602a4a10800 2023-08-11T15:51:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f94232e4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a49597e0 con 0x5602a4a0fe40 2023-08-11T15:51:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940d79b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a15690 0x5602a4a15a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:53388/0 (socket says 172.21.15.32:53388) 2023-08-11T15:51:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 -- 172.21.15.32:0/2279117184 learned_addr learned my addr 172.21.15.32:0/2279117184 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f93f40a01c0 0x7f93f40a2680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9408198300 tx=0x7f93f8009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940df9c700 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a10800 0x5602a4a1c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 -- 172.21.15.32:0/2279117184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a10800 msgr2=0x5602a4a1c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a10800 0x5602a4a1c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 -- 172.21.15.32:0/2279117184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a15690 msgr2=0x5602a4a15a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.445+0000 7f940cf9a700 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a15690 0x5602a4a15a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.446+0000 7f940cf9a700 1 -- 172.21.15.32:0/2279117184 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602a47d95d0 con 0x5602a4a0fe40 2023-08-11T15:51:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.446+0000 7f940d79b700 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a15690 0x5602a4a15a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.446+0000 7f940df9c700 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a10800 0x5602a4a1c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.446+0000 7f940cf9a700 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a102c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f93f8004250 tx=0x7f93f801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c376e4525854fc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.446+0000 7f93f37fe700 1 -- 172.21.15.32:0/2279117184 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8003d10 con 0x5602a4a0fe40 2023-08-11T15:51:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.446+0000 7f93f37fe700 1 -- 172.21.15.32:0/2279117184 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f8003e70 con 0x5602a4a0fe40 2023-08-11T15:51:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.446+0000 7f93f37fe700 1 -- 172.21.15.32:0/2279117184 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f801c720 con 0x5602a4a0fe40 2023-08-11T15:51:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.447+0000 7f94232e4b80 1 -- 172.21.15.32:0/2279117184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 msgr2=0x5602a4a102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.447+0000 7f94232e4b80 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a102c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f93f8004250 tx=0x7f93f801c000 comp rx=0 tx=0).stop 2023-08-11T15:51:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 -- 172.21.15.32:0/2279117184 shutdown_connections 2023-08-11T15:51:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a15690 0x5602a4a15a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a10800 0x5602a4a1c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 --2- 172.21.15.32:0/2279117184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a102c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 -- 172.21.15.32:0/2279117184 >> 172.21.15.32:0/2279117184 conn(0x5602a49348f0 msgr2=0x5602a4a01e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 -- 172.21.15.32:0/2279117184 shutdown_connections 2023-08-11T15:51:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 -- 172.21.15.32:0/2279117184 wait complete. 2023-08-11T15:51:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.448+0000 7f94232e4b80 1 -- start start 2023-08-11T15:51:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 1 Processor -- start 2023-08-11T15:51:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a1f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a10800 0x5602a4a1f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a19550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a48f7a60 con 0x5602a4a10800 2023-08-11T15:51:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a48f7760 con 0x5602a4a15690 2023-08-11T15:51:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a47db8c0 con 0x5602a4a0fe40 2023-08-11T15:51:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.449+0000 7f94232e4b80 10 client.0 _ll_register_callbacks cb 0x5602a48faa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.450+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a10800 0x5602a4a1f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.450+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a10800 0x5602a4a1f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57526/0 (socket says 172.21.15.32:57526) 2023-08-11T15:51:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.450+0000 7f940cf9a700 1 -- 172.21.15.32:0/4015237938 learned_addr learned my addr 172.21.15.32:0/4015237938 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.450+0000 7f940df9c700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a19550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.450+0000 7f940d79b700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a1f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f940df9c700 1 -- 172.21.15.32:0/4015237938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 msgr2=0x5602a4a1f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f940df9c700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a1f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f940df9c700 1 -- 172.21.15.32:0/4015237938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a10800 msgr2=0x5602a4a1f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f940df9c700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a10800 0x5602a4a1f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f940df9c700 1 -- 172.21.15.32:0/4015237938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940810e020 con 0x5602a4a15690 2023-08-11T15:51:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f940d79b700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a1f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f940df9c700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a19550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94000c8450 tx=0x7f9400097d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea32c3e7ad07f4c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f93f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:51:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f93f0ff9700 1 -- 172.21.15.32:0/4015237938 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94000c88d0 con 0x5602a4a15690 2023-08-11T15:51:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f93f0ff9700 1 -- 172.21.15.32:0/4015237938 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9400098b10 con 0x5602a4a15690 2023-08-11T15:51:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602a49fa7e0 con 0x5602a4a15690 2023-08-11T15:51:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.451+0000 7f93f0ff9700 1 -- 172.21.15.32:0/4015237938 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94000b05c0 con 0x5602a4a15690 2023-08-11T15:51:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 10 client.25757 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f93f8017050 con 0x5602a4a15690 2023-08-11T15:51:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f93f0ff9700 1 -- 172.21.15.32:0/4015237938 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94000b4080 con 0x5602a4a15690 2023-08-11T15:51:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f93f0ff9700 1 -- 172.21.15.32:0/4015237938 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9400022250 con 0x5602a4a15690 2023-08-11T15:51:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 20 client.25757 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a297c0 0x5602a4a2bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 4 client.25757 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:51:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602a49d6480 con 0x5602a4a297c0 2023-08-11T15:51:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.452+0000 7f940d79b700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a297c0 0x5602a4a2bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.453+0000 7f940d79b700 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a297c0 0x5602a4a2bbe0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe10b988000af2 server_cookie=4582edae177a0c32 in_seq=0 out_seq=0 2023-08-11T15:51:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.453+0000 7f93f0ff9700 10 client.25757 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f93f0ff9700 1 -- 172.21.15.32:0/4015237938 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5602a49d6480 con 0x5602a4a297c0 2023-08-11T15:51:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f93f0ff9700 10 client.25757 handle_command_reply: tid=0 2023-08-11T15:51:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 client.25757 shutdown 2023-08-11T15:51:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 msgr2=0x5602a4a19550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a19550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94000c8450 tx=0x7f9400097d70 comp rx=0 tx=0).stop 2023-08-11T15:51:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 shutdown_connections 2023-08-11T15:51:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a297c0 0x5602a4a2bbe0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a19550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a0fe40 0x5602a4a1f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f93f0ff9700 0 client.25757 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 --2- 172.21.15.32:0/4015237938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a10800 0x5602a4a1f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 >> 172.21.15.32:0/4015237938 conn(0x5602a49348f0 msgr2=0x5602a4a015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 shutdown_connections 2023-08-11T15:51:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.454+0000 7f94232e4b80 1 -- 172.21.15.32:0/4015237938 wait complete. 2023-08-11T15:51:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.455+0000 7f94232e4b80 20 client.25757 trim_cache size 0 max 16384 2023-08-11T15:51:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.471+0000 7f94232e4b80 1 Processor -- start 2023-08-11T15:51:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.471+0000 7f94232e4b80 1 -- start start 2023-08-11T15:51:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a01d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a1eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f94232e4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a4a14350 con 0x5602a4a0fe40 2023-08-11T15:51:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f94232e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a4a132c0 con 0x5602a4a15690 2023-08-11T15:51:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f94232e4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a4a02920 con 0x5602a4a10800 2023-08-11T15:51:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940d79b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a01d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940df9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a1eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940d79b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a171e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57528/0 (socket says 172.21.15.32:57528) 2023-08-11T15:51:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940d79b700 1 -- 172.21.15.32:0/3217211866 learned_addr learned my addr 172.21.15.32:0/3217211866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940df9c700 1 -- 172.21.15.32:0/3217211866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 msgr2=0x5602a4a01d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940df9c700 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a01d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940df9c700 1 -- 172.21.15.32:0/3217211866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 msgr2=0x5602a4a171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940df9c700 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a171e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940df9c700 1 -- 172.21.15.32:0/3217211866 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602a4785d80 con 0x5602a4a15690 2023-08-11T15:51:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.472+0000 7f940d79b700 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.473+0000 7f940df9c700 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a1eee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9400094e80 tx=0x7f940009c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b4cbc8b28b8198 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.473+0000 7f93f3fff700 1 -- 172.21.15.32:0/3217211866 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400098410 con 0x5602a4a15690 2023-08-11T15:51:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.473+0000 7f93f3fff700 1 -- 172.21.15.32:0/3217211866 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602a49fa7e0 con 0x5602a4a15690 2023-08-11T15:51:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.473+0000 7f93f3fff700 1 -- 172.21.15.32:0/3217211866 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400022250 con 0x5602a4a15690 2023-08-11T15:51:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 -- 172.21.15.32:0/3217211866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 msgr2=0x5602a4a1eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a1eee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9400094e80 tx=0x7f940009c120 comp rx=0 tx=0).stop 2023-08-11T15:51:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 -- 172.21.15.32:0/3217211866 shutdown_connections 2023-08-11T15:51:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a1eee0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 --2- 172.21.15.32:0/3217211866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a01d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 -- 172.21.15.32:0/3217211866 >> 172.21.15.32:0/3217211866 conn(0x5602a49348f0 msgr2=0x5602a4a19e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 -- 172.21.15.32:0/3217211866 shutdown_connections 2023-08-11T15:51:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 -- 172.21.15.32:0/3217211866 wait complete. 2023-08-11T15:51:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.474+0000 7f94232e4b80 1 -- start start 2023-08-11T15:51:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 1 Processor -- start 2023-08-11T15:51:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a23b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a4a242e0 con 0x5602a4a0fe40 2023-08-11T15:51:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a4a24450 con 0x5602a4a15690 2023-08-11T15:51:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a4a7c9e0 con 0x5602a4a10800 2023-08-11T15:51:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.475+0000 7f94232e4b80 10 client.0 _ll_register_callbacks cb 0x7f9400097f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940d79b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a23b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a23b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:53420/0 (socket says 172.21.15.32:53420) 2023-08-11T15:51:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940cf9a700 1 -- 172.21.15.32:0/1146265553 learned_addr learned my addr 172.21.15.32:0/1146265553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940df9c700 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940df9c700 1 -- 172.21.15.32:0/1146265553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 msgr2=0x5602a4a23b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940df9c700 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a23b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940df9c700 1 -- 172.21.15.32:0/1146265553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 msgr2=0x5602a4a235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940df9c700 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940df9c700 1 -- 172.21.15.32:0/1146265553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940400be60 con 0x5602a4a15690 2023-08-11T15:51:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.476+0000 7f940df9c700 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a7c470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5602a4a041a0 tx=0x7f9400094110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7627dd7c8dde7cee server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f93f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:51:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f93f0ff9700 1 -- 172.21.15.32:0/1146265553 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940009c240 con 0x5602a4a15690 2023-08-11T15:51:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f93f0ff9700 1 -- 172.21.15.32:0/1146265553 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9400098b10 con 0x5602a4a15690 2023-08-11T15:51:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93f8003e70 con 0x5602a4a15690 2023-08-11T15:51:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f93f0ff9700 1 -- 172.21.15.32:0/1146265553 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94000a0220 con 0x5602a4a15690 2023-08-11T15:51:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f94232e4b80 10 client.25763 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f93f8017050 con 0x5602a4a15690 2023-08-11T15:51:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.477+0000 7f93f0ff9700 1 -- 172.21.15.32:0/1146265553 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f8003e70 con 0x5602a4a15690 2023-08-11T15:51:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f93f0ff9700 1 -- 172.21.15.32:0/1146265553 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f940009fd90 con 0x5602a4a15690 2023-08-11T15:51:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f94232e4b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f94232e4b80 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f94232e4b80 20 client.25763 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f94232e4b80 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a7dc40 0x5602a4a80060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f94232e4b80 4 client.25763 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:51:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5602a49527c0 con 0x5602a4a7dc40 2023-08-11T15:51:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f940d79b700 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a7dc40 0x5602a4a80060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f940d79b700 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a7dc40 0x5602a4a80060 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ade82d0a1442739 server_cookie=4103ab7fabd31aa4 in_seq=0 out_seq=0 2023-08-11T15:51:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.478+0000 7f93f0ff9700 10 client.25763 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:12.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:12 smithi032 ceph-mon[97103]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.8 MiB/s wr, 582 op/s 2023-08-11T15:51:12.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:12 smithi192 ceph-mon[130806]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.8 MiB/s wr, 582 op/s 2023-08-11T15:51:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f93f0ff9700 1 -- 172.21.15.32:0/1146265553 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5602a49527c0 con 0x5602a4a7dc40 2023-08-11T15:51:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f93f0ff9700 10 client.25763 handle_command_reply: tid=0 2023-08-11T15:51:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 client.25763 shutdown 2023-08-11T15:51:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 msgr2=0x5602a4a7c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a7c470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5602a4a041a0 tx=0x7f9400094110 comp rx=0 tx=0).stop 2023-08-11T15:51:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 shutdown_connections 2023-08-11T15:51:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5602a4a7dc40 0x5602a4a80060 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5602a4a0fe40 0x5602a4a235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602a4a15690 0x5602a4a7c470 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f93f0ff9700 0 client.25763 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 --2- 172.21.15.32:0/1146265553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602a4a10800 0x5602a4a23b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 >> 172.21.15.32:0/1146265553 conn(0x5602a49348f0 msgr2=0x5602a4a1ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 shutdown_connections 2023-08-11T15:51:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.707+0000 7f94232e4b80 1 -- 172.21.15.32:0/1146265553 wait complete. 2023-08-11T15:51:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.708+0000 7f94232e4b80 20 client.25763 trim_cache size 0 max 16384 2023-08-11T15:51:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:51:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:51:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:51:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:51:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.708+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f93f40a01c0 msgr2=0x7f93f40a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.708+0000 7f940f9ff700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f93f40a01c0 0x7f93f40a2680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9408198300 tx=0x7f93f8009260 comp rx=0 tx=0).stop 2023-08-11T15:51:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.708+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 msgr2=0x7f94081968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.708+0000 7f940f9ff700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f94081968c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9400007ee0 tx=0x7f940000d3b0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 shutdown_connections 2023-08-11T15:51:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f93f40a01c0 0x7f93f40a2680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9408067d10 0x7f940819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94081068d0 0x7f94081968c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 --2- 172.21.15.32:0/3209227725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94080686d0 0x7f940819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 >> 172.21.15.32:0/3209227725 conn(0x7f9408075250 msgr2=0x7f94080fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 shutdown_connections 2023-08-11T15:51:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:12.709+0000 7f940f9ff700 1 -- 172.21.15.32:0/3209227725 wait complete. 2023-08-11T15:51:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:12 smithi134 ceph-mon[130447]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.8 MiB/s wr, 582 op/s 2023-08-11T15:51:12.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 498468 12.57 MB/sec execute 88 sec latency 154.944 ms 2023-08-11T15:51:13.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 501183 12.62 MB/sec execute 89 sec latency 16.685 ms 2023-08-11T15:51:14.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:14 smithi032 ceph-mon[97103]: pgmap v142: 129 pgs: 129 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.6 MiB/s wr, 660 op/s 2023-08-11T15:51:14.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:14 smithi192 ceph-mon[130806]: pgmap v142: 129 pgs: 129 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.6 MiB/s wr, 660 op/s 2023-08-11T15:51:14.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:14 smithi134 ceph-mon[130447]: pgmap v142: 129 pgs: 129 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.6 MiB/s wr, 660 op/s 2023-08-11T15:51:14.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 503141 12.54 MB/sec execute 90 sec latency 19.673 ms 2023-08-11T15:51:15.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 505226 12.56 MB/sec execute 91 sec latency 18.900 ms 2023-08-11T15:51:16.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:16 smithi032 ceph-mon[97103]: pgmap v143: 129 pgs: 129 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2023-08-11T15:51:16.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:16 smithi192 ceph-mon[130806]: pgmap v143: 129 pgs: 129 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2023-08-11T15:51:16.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:16 smithi134 ceph-mon[130447]: pgmap v143: 129 pgs: 129 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2023-08-11T15:51:16.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 508033 12.61 MB/sec execute 92 sec latency 11.634 ms 2023-08-11T15:51:17.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 509553 12.54 MB/sec execute 93 sec latency 73.443 ms 2023-08-11T15:51:18.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:17 smithi134 ceph-mon[130447]: pgmap v144: 129 pgs: 129 active+clean; 56 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 376 op/s 2023-08-11T15:51:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:17 smithi032 ceph-mon[97103]: pgmap v144: 129 pgs: 129 active+clean; 56 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 376 op/s 2023-08-11T15:51:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:17 smithi192 ceph-mon[130806]: pgmap v144: 129 pgs: 129 active+clean; 56 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 376 op/s 2023-08-11T15:51:18.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 512115 12.56 MB/sec execute 94 sec latency 11.788 ms 2023-08-11T15:51:19.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 514591 12.56 MB/sec execute 95 sec latency 16.145 ms 2023-08-11T15:51:20.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:20 smithi032 ceph-mon[97103]: pgmap v145: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2023-08-11T15:51:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:20 smithi192 ceph-mon[130806]: pgmap v145: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2023-08-11T15:51:20.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:20 smithi134 ceph-mon[130447]: pgmap v145: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2023-08-11T15:51:20.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 516656 12.55 MB/sec execute 96 sec latency 14.812 ms 2023-08-11T15:51:21.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 519133 12.56 MB/sec execute 97 sec latency 27.182 ms 2023-08-11T15:51:22.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:22 smithi032 ceph-mon[97103]: pgmap v146: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 268 op/s 2023-08-11T15:51:22.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:22 smithi192 ceph-mon[130806]: pgmap v146: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 268 op/s 2023-08-11T15:51:22.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:22 smithi134 ceph-mon[130447]: pgmap v146: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 268 op/s 2023-08-11T15:51:22.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 521274 12.52 MB/sec execute 98 sec latency 16.588 ms 2023-08-11T15:51:23.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 523368 12.52 MB/sec execute 99 sec latency 27.224 ms 2023-08-11T15:51:24.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:24 smithi032 ceph-mon[97103]: pgmap v147: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-08-11T15:51:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:24 smithi192 ceph-mon[130806]: pgmap v147: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-08-11T15:51:24.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:24 smithi134 ceph-mon[130447]: pgmap v147: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 341 op/s 2023-08-11T15:51:24.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 526005 12.57 MB/sec execute 100 sec latency 22.705 ms 2023-08-11T15:51:25.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 527964 12.50 MB/sec execute 101 sec latency 27.726 ms 2023-08-11T15:51:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 -- 172.21.15.32:0/831562536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f810d1a0 msgr2=0x7f58f810f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 --2- 172.21.15.32:0/831562536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f810d1a0 0x7f58f810f590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009f90 tx=0x7f58f000b330 comp rx=0 tx=0).stop 2023-08-11T15:51:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 -- 172.21.15.32:0/831562536 shutdown_connections 2023-08-11T15:51:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 --2- 172.21.15.32:0/831562536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f810d1a0 0x7f58f810f590 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 --2- 172.21.15.32:0/831562536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f8072cd0 0x7f58f80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 --2- 172.21.15.32:0/831562536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58f80735f0 0x7f58f810cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 -- 172.21.15.32:0/831562536 >> 172.21.15.32:0/831562536 conn(0x7f58f80fc3f0 msgr2=0x7f58f80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 -- 172.21.15.32:0/831562536 shutdown_connections 2023-08-11T15:51:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.865+0000 7f58fcdff700 1 -- 172.21.15.32:0/831562536 wait complete. 2023-08-11T15:51:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.866+0000 7f58fcdff700 1 Processor -- start 2023-08-11T15:51:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.866+0000 7f58fcdff700 1 -- start start 2023-08-11T15:51:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58fcdff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f8072cd0 0x7f58f819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58fcdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58f80735f0 0x7f58f819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58fcdff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f810d1a0 0x7f58f8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58fcdff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81991b0 con 0x7f58f8072cd0 2023-08-11T15:51:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58fcdff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8199320 con 0x7f58f810d1a0 2023-08-11T15:51:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58fcdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81994e0 con 0x7f58f80735f0 2023-08-11T15:51:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f8072cd0 0x7f58f819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f810d1a0 0x7f58f8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f810d1a0 0x7f58f8198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:45148/0 (socket says 172.21.15.32:45148) 2023-08-11T15:51:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 -- 172.21.15.32:0/3067440465 learned_addr learned my addr 172.21.15.32:0/3067440465 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58f80735f0 0x7f58f819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 -- 172.21.15.32:0/3067440465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58f80735f0 msgr2=0x7f58f819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58f80735f0 0x7f58f819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 -- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f8072cd0 msgr2=0x7f58f819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f8072cd0 0x7f58f819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.867+0000 7f58f6d9d700 1 -- 172.21.15.32:0/3067440465 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0009c40 con 0x7f58f810d1a0 2023-08-11T15:51:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.868+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f810d1a0 0x7f58f8198b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b6c0 tx=0x7f58f0009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.868+0000 7f58e77fe700 1 -- 172.21.15.32:0/3067440465 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000e040 con 0x7f58f810d1a0 2023-08-11T15:51:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.868+0000 7f58e77fe700 1 -- 172.21.15.32:0/3067440465 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f000bbf0 con 0x7f58f810d1a0 2023-08-11T15:51:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.868+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f8199760 con 0x7f58f810d1a0 2023-08-11T15:51:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.868+0000 7f58e77fe700 1 -- 172.21.15.32:0/3067440465 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0005040 con 0x7f58f810d1a0 2023-08-11T15:51:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.868+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a5cd0 con 0x7f58f810d1a0 2023-08-11T15:51:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.872+0000 7f58e77fe700 1 -- 172.21.15.32:0/3067440465 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f58f001d470 con 0x7f58f810d1a0 2023-08-11T15:51:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.873+0000 7f58e77fe700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f58e00a00c0 0x7f58e00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.873+0000 7f59106a7b80 1 Processor -- start 2023-08-11T15:51:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.873+0000 7f58e77fe700 1 -- 172.21.15.32:0/3067440465 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f58f00ccd10 con 0x7f58f810d1a0 2023-08-11T15:51:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f58f659c700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f58e00a00c0 0x7f58e00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f59106a7b80 1 -- start start 2023-08-11T15:51:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0630 0x56165a3b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0ff0 0x56165a3bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f59106a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a3095d0 con 0x56165a3b0630 2023-08-11T15:51:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f59106a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a306ef0 con 0x56165a3b5e80 2023-08-11T15:51:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f59106a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a3027e0 con 0x56165a3b0ff0 2023-08-11T15:51:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0630 0x56165a3b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f58f659c700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f58e00a00c0 0x7f58e00a2580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009c60 tx=0x7f58e8009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f58f659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.874+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0630 0x56165a3b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55730/0 (socket says 172.21.15.32:55730) 2023-08-11T15:51:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:45150/0 (socket says 172.21.15.32:45150) 2023-08-11T15:51:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f5d9b700 1 -- 172.21.15.32:0/3800431868 learned_addr learned my addr 172.21.15.32:0/3800431868 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0ff0 0x56165a3bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f659c700 1 -- 172.21.15.32:0/3800431868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0ff0 msgr2=0x56165a3bcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f659c700 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0ff0 0x56165a3bcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f659c700 1 -- 172.21.15.32:0/3800431868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0630 msgr2=0x56165a3b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f659c700 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0630 0x56165a3b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f659c700 1 -- 172.21.15.32:0/3800431868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56165a1825d0 con 0x56165a3b5e80 2023-08-11T15:51:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0ff0 0x56165a3bcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58f659c700 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58e8005950 tx=0x7f58e8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6458e408fe6434c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58e4ff9700 1 -- 172.21.15.32:0/3800431868 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800ecf0 con 0x56165a3b5e80 2023-08-11T15:51:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58e4ff9700 1 -- 172.21.15.32:0/3800431868 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e8021070 con 0x56165a3b5e80 2023-08-11T15:51:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.875+0000 7f58e4ff9700 1 -- 172.21.15.32:0/3800431868 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e801d410 con 0x56165a3b5e80 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 -- 172.21.15.32:0/3800431868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 msgr2=0x56165a3b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58e8005950 tx=0x7f58e8014000 comp rx=0 tx=0).stop 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 -- 172.21.15.32:0/3800431868 shutdown_connections 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0ff0 0x56165a3bcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b6260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 --2- 172.21.15.32:0/3800431868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0630 0x56165a3b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 -- 172.21.15.32:0/3800431868 >> 172.21.15.32:0/3800431868 conn(0x56165a2dd8f0 msgr2=0x56165a3aae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.876+0000 7f59106a7b80 1 -- 172.21.15.32:0/3800431868 shutdown_connections 2023-08-11T15:51:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.877+0000 7f59106a7b80 1 -- 172.21.15.32:0/3800431868 wait complete. 2023-08-11T15:51:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.877+0000 7f59106a7b80 1 -- start start 2023-08-11T15:51:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.877+0000 7f59106a7b80 1 Processor -- start 2023-08-11T15:51:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a3bff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a3b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f59106a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a2a0a60 con 0x56165a3b5e80 2023-08-11T15:51:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f59106a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a2a0760 con 0x56165a3b0ff0 2023-08-11T15:51:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f59106a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a1848c0 con 0x56165a3b0630 2023-08-11T15:51:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f59106a7b80 10 client.0 _ll_register_callbacks cb 0x56165a2a3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f58f659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a3bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f58f659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3bfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:58960/0 (socket says 172.21.15.32:58960) 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f58f659c700 1 -- 172.21.15.32:0/1120852203 learned_addr learned my addr 172.21.15.32:0/1120852203 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.878+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a3b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58f659c700 1 -- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 msgr2=0x56165a3bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58f659c700 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a3bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58f659c700 1 -- 172.21.15.32:0/1120852203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 msgr2=0x56165a3b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58f659c700 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a3b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58f659c700 1 -- 172.21.15.32:0/1120852203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a5cd0 con 0x56165a3b0630 2023-08-11T15:51:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a3bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58f659c700 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3bfa40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58e80053b0 tx=0x7f58e80055e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e1c5874f5a8b8a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:51:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58ddffb700 1 -- 172.21.15.32:0/1120852203 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e802a040 con 0x56165a3b0630 2023-08-11T15:51:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56165a3a37e0 con 0x56165a3b0630 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.879+0000 7f58ddffb700 1 -- 172.21.15.32:0/1120852203 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f58e8004750 con 0x56165a3b0630 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f58ddffb700 1 -- 172.21.15.32:0/1120852203 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e8004910 con 0x56165a3b0630 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 10 client.25777 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f58e8017070 con 0x56165a3b0630 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f58ddffb700 1 -- 172.21.15.32:0/1120852203 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56165a3a37e0 con 0x56165a3b0630 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f58ddffb700 1 -- 172.21.15.32:0/1120852203 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f58e8026a60 con 0x56165a3b0630 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 10 client.25777 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 20 client.25777 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a3c9e10 0x56165a3cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 4 client.25777 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56165a37f480 con 0x56165a3c9e10 2023-08-11T15:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.880+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a3c9e10 0x56165a3cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.881+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a3c9e10 0x56165a3cc230 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef680ea173bb1e3 server_cookie=2e09ee6fe3e2eee6 in_seq=0 out_seq=0 2023-08-11T15:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.881+0000 7f58ddffb700 10 client.25777 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f58ddffb700 1 -- 172.21.15.32:0/1120852203 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56165a37f480 con 0x56165a3c9e10 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f58ddffb700 10 client.25777 handle_command_reply: tid=0 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 client.25777 shutdown 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 msgr2=0x56165a3bfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3bfa40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58e80053b0 tx=0x7f58e80055e0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 shutdown_connections 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a3c9e10 0x56165a3cc230 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a3b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3bfa40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f58ddffb700 0 client.25777 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 --2- 172.21.15.32:0/1120852203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a3bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 >> 172.21.15.32:0/1120852203 conn(0x56165a2dd8f0 msgr2=0x56165a3aa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 shutdown_connections 2023-08-11T15:51:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.882+0000 7f59106a7b80 1 -- 172.21.15.32:0/1120852203 wait complete. 2023-08-11T15:51:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.883+0000 7f59106a7b80 20 client.25777 trim_cache size 0 max 16384 2023-08-11T15:51:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.898+0000 7f59106a7b80 1 Processor -- start 2023-08-11T15:51:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f59106a7b80 1 -- start start 2023-08-11T15:51:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0ff0 0x56165a3d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f59106a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a3b4b10 con 0x56165a3b0ff0 2023-08-11T15:51:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f59106a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a3b3ab0 con 0x56165a3b5e80 2023-08-11T15:51:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f59106a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a3cc890 con 0x56165a3b0630 2023-08-11T15:51:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0ff0 0x56165a3d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0ff0 0x56165a3d0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55736/0 (socket says 172.21.15.32:55736) 2023-08-11T15:51:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f58f5d9b700 1 -- 172.21.15.32:0/960345792 learned_addr learned my addr 172.21.15.32:0/960345792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.899+0000 7f58f659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f6d9d700 1 -- 172.21.15.32:0/960345792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 msgr2=0x56165a3b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f6d9d700 1 -- 172.21.15.32:0/960345792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0ff0 msgr2=0x56165a3d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0ff0 0x56165a3d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f6d9d700 1 -- 172.21.15.32:0/960345792 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56165a12ed80 con 0x56165a3b5e80 2023-08-11T15:51:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f659c700 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0ff0 0x56165a3d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b5630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f58f00ca150 tx=0x7f58f001dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563b473fc299b78d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58e57fa700 1 -- 172.21.15.32:0/960345792 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f009a8a0 con 0x56165a3b5e80 2023-08-11T15:51:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58e57fa700 1 -- 172.21.15.32:0/960345792 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f009a1a0 con 0x56165a3b5e80 2023-08-11T15:51:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.900+0000 7f58e57fa700 1 -- 172.21.15.32:0/960345792 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00749c0 con 0x56165a3b5e80 2023-08-11T15:51:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 -- 172.21.15.32:0/960345792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 msgr2=0x56165a3b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b5630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f58f00ca150 tx=0x7f58f001dad0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 -- 172.21.15.32:0/960345792 shutdown_connections 2023-08-11T15:51:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b5e80 0x56165a3b5630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a3b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 --2- 172.21.15.32:0/960345792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b0ff0 0x56165a3d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 -- 172.21.15.32:0/960345792 >> 172.21.15.32:0/960345792 conn(0x56165a2dd8f0 msgr2=0x56165a3b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 -- 172.21.15.32:0/960345792 shutdown_connections 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.901+0000 7f59106a7b80 1 -- 172.21.15.32:0/960345792 wait complete. 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.902+0000 7f59106a7b80 1 -- start start 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.902+0000 7f59106a7b80 1 Processor -- start 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a411c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a412160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f59106a7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a41cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f59106a7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a41cff0 con 0x56165a3b5e80 2023-08-11T15:51:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f59106a7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a41d130 con 0x56165a3b0ff0 2023-08-11T15:51:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f59106a7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165a41d270 con 0x56165a3b0630 2023-08-11T15:51:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f59106a7b80 10 client.0 _ll_register_callbacks cb 0x7f58ec009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a412160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a411c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a412160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:45174/0 (socket says 172.21.15.32:45174) 2023-08-11T15:51:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 -- 172.21.15.32:0/105596994 learned_addr learned my addr 172.21.15.32:0/105596994 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f6d9d700 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a41cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 -- 172.21.15.32:0/105596994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 msgr2=0x56165a411c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a411c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 -- 172.21.15.32:0/105596994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 msgr2=0x56165a41cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a41cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f5d9b700 1 -- 172.21.15.32:0/105596994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8017070 con 0x56165a3b0ff0 2023-08-11T15:51:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.903+0000 7f58f659c700 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a411c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f58f5d9b700 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a412160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56165a3abe90 tx=0x7f58ec009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea6cf70905ecc633 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f58ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:51:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f58ddffb700 1 -- 172.21.15.32:0/105596994 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec017030 con 0x56165a3b0ff0 2023-08-11T15:51:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58e8021070 con 0x56165a3b0ff0 2023-08-11T15:51:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f58ddffb700 1 -- 172.21.15.32:0/105596994 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f58ec01eac0 con 0x56165a3b0ff0 2023-08-11T15:51:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f58ddffb700 1 -- 172.21.15.32:0/105596994 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ec012e80 con 0x56165a3b0ff0 2023-08-11T15:51:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f59106a7b80 10 client.25781 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.904+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f58ec012040 con 0x56165a3b0ff0 2023-08-11T15:51:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f58ddffb700 1 -- 172.21.15.32:0/105596994 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e8021070 con 0x56165a3b0ff0 2023-08-11T15:51:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f58ddffb700 1 -- 172.21.15.32:0/105596994 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f58ec026420 con 0x56165a3b0ff0 2023-08-11T15:51:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f59106a7b80 10 client.25781 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f59106a7b80 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f59106a7b80 20 client.25781 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f59106a7b80 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a41ea30 0x56165a420e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f59106a7b80 4 client.25781 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:51:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56165a2fb7c0 con 0x56165a41ea30 2023-08-11T15:51:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.905+0000 7f58f659c700 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a41ea30 0x56165a420e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.906+0000 7f58f659c700 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a41ea30 0x56165a420e50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1706d8c8a8e9de60 server_cookie=3aa604dae87a37c3 in_seq=0 out_seq=0 2023-08-11T15:51:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:25.906+0000 7f58ddffb700 10 client.25781 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f58ddffb700 1 -- 172.21.15.32:0/105596994 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56165a2fb7c0 con 0x56165a41ea30 2023-08-11T15:51:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f58ddffb700 10 client.25781 handle_command_reply: tid=0 2023-08-11T15:51:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 client.25781 shutdown 2023-08-11T15:51:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 msgr2=0x56165a412160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a412160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56165a3abe90 tx=0x7f58ec009690 comp rx=0 tx=0).stop 2023-08-11T15:51:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 shutdown_connections 2023-08-11T15:51:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x56165a41ea30 0x56165a420e50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56165a3b5e80 0x56165a41cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56165a3b0630 0x56165a411c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 --2- 172.21.15.32:0/105596994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56165a3b0ff0 0x56165a412160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 >> 172.21.15.32:0/105596994 conn(0x56165a2dd8f0 msgr2=0x56165a3af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.179+0000 7f58ddffb700 0 client.25781 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.180+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 shutdown_connections 2023-08-11T15:51:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.181+0000 7f59106a7b80 1 -- 172.21.15.32:0/105596994 wait complete. 2023-08-11T15:51:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.181+0000 7f59106a7b80 20 client.25781 trim_cache size 0 max 16384 2023-08-11T15:51:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:51:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:51:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:51:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:51:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f58e00a00c0 msgr2=0x7f58e00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f58e00a00c0 0x7f58e00a2580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009c60 tx=0x7f58e8009260 comp rx=0 tx=0).stop 2023-08-11T15:51:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f810d1a0 msgr2=0x7f58f8198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f810d1a0 0x7f58f8198b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b6c0 tx=0x7f58f0009510 comp rx=0 tx=0).stop 2023-08-11T15:51:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 shutdown_connections 2023-08-11T15:51:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f58e00a00c0 0x7f58e00a2580 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f8072cd0 0x7f58f819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f810d1a0 0x7f58f8198b40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 --2- 172.21.15.32:0/3067440465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58f80735f0 0x7f58f819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 >> 172.21.15.32:0/3067440465 conn(0x7f58f80fc3f0 msgr2=0x7f58f810f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.182+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 shutdown_connections 2023-08-11T15:51:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:26.183+0000 7f58fcdff700 1 -- 172.21.15.32:0/3067440465 wait complete. 2023-08-11T15:51:26.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:26 smithi032 ceph-mon[97103]: pgmap v148: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.5 MiB/s wr, 264 op/s 2023-08-11T15:51:26.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:26 smithi192 ceph-mon[130806]: pgmap v148: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.5 MiB/s wr, 264 op/s 2023-08-11T15:51:26.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:26 smithi134 ceph-mon[130447]: pgmap v148: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.5 MiB/s wr, 264 op/s 2023-08-11T15:51:26.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 529723 12.51 MB/sec execute 102 sec latency 123.719 ms 2023-08-11T15:51:27.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 531782 12.46 MB/sec execute 103 sec latency 40.746 ms 2023-08-11T15:51:28.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:27 smithi134 ceph-mon[130447]: pgmap v149: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 444 op/s 2023-08-11T15:51:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:27 smithi032 ceph-mon[97103]: pgmap v149: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 444 op/s 2023-08-11T15:51:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:27 smithi192 ceph-mon[130806]: pgmap v149: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 444 op/s 2023-08-11T15:51:28.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 533677 12.46 MB/sec execute 104 sec latency 114.641 ms 2023-08-11T15:51:29.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 535712 12.44 MB/sec execute 105 sec latency 181.031 ms 2023-08-11T15:51:30.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:30 smithi032 ceph-mon[97103]: pgmap v150: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 367 op/s 2023-08-11T15:51:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:30 smithi192 ceph-mon[130806]: pgmap v150: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 367 op/s 2023-08-11T15:51:30.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:30 smithi134 ceph-mon[130447]: pgmap v150: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 367 op/s 2023-08-11T15:51:30.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 537815 12.45 MB/sec execute 106 sec latency 13.841 ms 2023-08-11T15:51:31.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 540346 12.46 MB/sec execute 107 sec latency 24.613 ms 2023-08-11T15:51:32.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:32 smithi032 ceph-mon[97103]: pgmap v151: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 299 op/s 2023-08-11T15:51:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:32 smithi192 ceph-mon[130806]: pgmap v151: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 299 op/s 2023-08-11T15:51:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:32 smithi134 ceph-mon[130447]: pgmap v151: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 299 op/s 2023-08-11T15:51:32.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 542477 12.42 MB/sec execute 108 sec latency 22.311 ms 2023-08-11T15:51:33.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 544579 12.43 MB/sec execute 109 sec latency 16.653 ms 2023-08-11T15:51:34.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:34 smithi032 ceph-mon[97103]: pgmap v152: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2023-08-11T15:51:34.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:34 smithi192 ceph-mon[130806]: pgmap v152: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2023-08-11T15:51:34.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:34 smithi134 ceph-mon[130447]: pgmap v152: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2023-08-11T15:51:34.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 546878 12.43 MB/sec execute 110 sec latency 61.047 ms 2023-08-11T15:51:35.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 548846 12.41 MB/sec execute 111 sec latency 13.676 ms 2023-08-11T15:51:36.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:36 smithi032 ceph-mon[97103]: pgmap v153: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-08-11T15:51:36.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:36 smithi192 ceph-mon[130806]: pgmap v153: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-08-11T15:51:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:36 smithi134 ceph-mon[130447]: pgmap v153: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-08-11T15:51:36.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 549963 12.35 MB/sec execute 112 sec latency 448.335 ms 2023-08-11T15:51:37.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 551210 12.31 MB/sec execute 113 sec latency 58.669 ms 2023-08-11T15:51:38.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:37 smithi134 ceph-mon[130447]: pgmap v154: 129 pgs: 129 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 371 op/s 2023-08-11T15:51:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:37 smithi032 ceph-mon[97103]: pgmap v154: 129 pgs: 129 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 371 op/s 2023-08-11T15:51:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:37 smithi192 ceph-mon[130806]: pgmap v154: 129 pgs: 129 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 371 op/s 2023-08-11T15:51:38.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 553976 12.33 MB/sec execute 114 sec latency 12.284 ms 2023-08-11T15:51:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 -- 172.21.15.32:0/956298937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 msgr2=0x7fafcc100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 --2- 172.21.15.32:0/956298937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 0x7fafcc100d30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fafb80098f0 tx=0x7fafb8009c00 comp rx=0 tx=0).stop 2023-08-11T15:51:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 -- 172.21.15.32:0/956298937 shutdown_connections 2023-08-11T15:51:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 --2- 172.21.15.32:0/956298937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafcc101270 0x7fafcc10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 --2- 172.21.15.32:0/956298937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 0x7fafcc106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 --2- 172.21.15.32:0/956298937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 0x7fafcc100d30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 -- 172.21.15.32:0/956298937 >> 172.21.15.32:0/956298937 conn(0x7fafcc0fc380 msgr2=0x7fafcc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 -- 172.21.15.32:0/956298937 shutdown_connections 2023-08-11T15:51:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.360+0000 7fafd2dff700 1 -- 172.21.15.32:0/956298937 wait complete. 2023-08-11T15:51:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 Processor -- start 2023-08-11T15:51:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 -- start start 2023-08-11T15:51:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 0x7fafcc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafcc101270 0x7fafcc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 0x7fafcc196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc197050 con 0x7fafcc101270 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1971c0 con 0x7fafcc1068c0 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.361+0000 7fafd2dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc197380 con 0x7fafcc1008b0 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 0x7fafcc196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 0x7fafcc196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55510/0 (socket says 172.21.15.32:55510) 2023-08-11T15:51:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 -- 172.21.15.32:0/3545597424 learned_addr learned my addr 172.21.15.32:0/3545597424 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafcbfff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafcc101270 0x7fafcc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 0x7fafcc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 -- 172.21.15.32:0/3545597424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 msgr2=0x7fafcc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 0x7fafcc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 -- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafcc101270 msgr2=0x7fafcc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafcc101270 0x7fafcc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 -- 172.21.15.32:0/3545597424 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8009520 con 0x7fafcc1068c0 2023-08-11T15:51:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 0x7fafcc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.362+0000 7fafd139c700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 0x7fafcc196880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fafc400ef30 tx=0x7fafc400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.363+0000 7fafc9ffb700 1 -- 172.21.15.32:0/3545597424 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4018070 con 0x7fafcc1068c0 2023-08-11T15:51:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.363+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc197600 con 0x7fafcc1068c0 2023-08-11T15:51:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.363+0000 7fafc9ffb700 1 -- 172.21.15.32:0/3545597424 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc400f040 con 0x7fafcc1068c0 2023-08-11T15:51:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.363+0000 7fafc9ffb700 1 -- 172.21.15.32:0/3545597424 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc400cdb0 con 0x7fafcc1068c0 2023-08-11T15:51:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.363+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc1a3a30 con 0x7fafcc1068c0 2023-08-11T15:51:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.366+0000 7fafc9ffb700 1 -- 172.21.15.32:0/3545597424 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fafc4003a00 con 0x7fafcc1068c0 2023-08-11T15:51:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 Processor -- start 2023-08-11T15:51:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 -- start start 2023-08-11T15:51:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafc9ffb700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fafbc0a00c0 0x7fafbc0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c6703010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6703550 0x5617c6706de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c6707760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c66575d0 con 0x5617c6707320 2023-08-11T15:51:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c6654ef0 con 0x5617c6702c30 2023-08-11T15:51:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafe6680b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c66507e0 con 0x5617c6703550 2023-08-11T15:51:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.368+0000 7fafc9ffb700 1 -- 172.21.15.32:0/3545597424 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fafc40cb600 con 0x7fafcc1068c0 2023-08-11T15:51:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafd0b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c6703010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafcbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6703550 0x5617c6706de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafcbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6703550 0x5617c6706de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43154/0 (socket says 172.21.15.32:43154) 2023-08-11T15:51:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafcbfff700 1 -- 172.21.15.32:0/2793616391 learned_addr learned my addr 172.21.15.32:0/2793616391 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafcbfff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fafbc0a00c0 0x7fafbc0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafd139c700 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c6707760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafd0b9b700 1 -- 172.21.15.32:0/2793616391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6703550 msgr2=0x5617c6706de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6703550 0x5617c6706de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafd0b9b700 1 -- 172.21.15.32:0/2793616391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 msgr2=0x5617c6707760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c6707760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.369+0000 7fafd0b9b700 1 -- 172.21.15.32:0/2793616391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c64d05d0 con 0x5617c6702c30 2023-08-11T15:51:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.370+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c6703010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009c40 tx=0x7fafc0009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d36dd94dd3f723e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.370+0000 7fafcbfff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fafbc0a00c0 0x7fafbc0a2580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fafb80098c0 tx=0x7fafb8005dc0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.370+0000 7fafb6ffd700 1 -- 172.21.15.32:0/2793616391 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0011bf0 con 0x5617c6702c30 2023-08-11T15:51:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.370+0000 7fafb6ffd700 1 -- 172.21.15.32:0/2793616391 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc0011d50 con 0x5617c6702c30 2023-08-11T15:51:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.370+0000 7fafb6ffd700 1 -- 172.21.15.32:0/2793616391 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000f6a0 con 0x5617c6702c30 2023-08-11T15:51:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 -- 172.21.15.32:0/2793616391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 msgr2=0x5617c6703010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c6703010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009c40 tx=0x7fafc0009f50 comp rx=0 tx=0).stop 2023-08-11T15:51:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 -- 172.21.15.32:0/2793616391 shutdown_connections 2023-08-11T15:51:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c6707760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c6703010 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 --2- 172.21.15.32:0/2793616391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6703550 0x5617c6706de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 -- 172.21.15.32:0/2793616391 >> 172.21.15.32:0/2793616391 conn(0x5617c662b8f0 msgr2=0x5617c66f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 -- 172.21.15.32:0/2793616391 shutdown_connections 2023-08-11T15:51:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 -- 172.21.15.32:0/2793616391 wait complete. 2023-08-11T15:51:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.371+0000 7fafe6680b80 1 -- start start 2023-08-11T15:51:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 1 Processor -- start 2023-08-11T15:51:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c670ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6703550 0x5617c670bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c670c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c65eea60 con 0x5617c6703550 2023-08-11T15:51:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c65ee760 con 0x5617c6702c30 2023-08-11T15:51:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c64d28c0 con 0x5617c6707320 2023-08-11T15:51:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafe6680b80 10 client.0 _ll_register_callbacks cb 0x5617c65f1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.372+0000 7fafd0b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c670ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c670ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55534/0 (socket says 172.21.15.32:55534) 2023-08-11T15:51:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1931065355 learned_addr learned my addr 172.21.15.32:0/1931065355 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafcbfff700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6703550 0x5617c670bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd139c700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c670c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1931065355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 msgr2=0x5617c670c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c670c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1931065355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6703550 msgr2=0x5617c670bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6703550 0x5617c670bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1931065355 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00098a0 con 0x5617c6702c30 2023-08-11T15:51:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd139c700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c670c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:51:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c670ea00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafc0006010 tx=0x7fafc0012700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c68ac63867a871e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafabfff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:51:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.373+0000 7fafabfff700 1 -- 172.21.15.32:0/1931065355 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0004030 con 0x5617c6702c30 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617c66f17e0 con 0x5617c6702c30 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafabfff700 1 -- 172.21.15.32:0/1931065355 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fafc0004190 con 0x5617c6702c30 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafabfff700 1 -- 172.21.15.32:0/1931065355 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc0004350 con 0x5617c6702c30 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 10 client.25793 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5617c647cd80 con 0x5617c6702c30 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafabfff700 1 -- 172.21.15.32:0/1931065355 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617c66f17e0 con 0x5617c6702c30 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafabfff700 1 -- 172.21.15.32:0/1931065355 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fafc002c060 con 0x5617c6702c30 2023-08-11T15:51:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 10 client.25793 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 10 client.25793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 20 client.25793 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c671c180 0x5617c671e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 4 client.25793 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.374+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617c66cd480 con 0x5617c671c180 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.375+0000 7fafcbfff700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c671c180 0x5617c671e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.375+0000 7fafcbfff700 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c671c180 0x5617c671e5a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4bf18f20904d93 server_cookie=e7929aa775882a97 in_seq=0 out_seq=0 2023-08-11T15:51:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.375+0000 7fafabfff700 10 client.25793 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafabfff700 1 -- 172.21.15.32:0/1931065355 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5617c66cd480 con 0x5617c671c180 2023-08-11T15:51:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafabfff700 10 client.25793 handle_command_reply: tid=0 2023-08-11T15:51:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 client.25793 shutdown 2023-08-11T15:51:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 msgr2=0x5617c670ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c670ea00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafc0006010 tx=0x7fafc0012700 comp rx=0 tx=0).stop 2023-08-11T15:51:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 shutdown_connections 2023-08-11T15:51:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c671c180 0x5617c671e5a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c670c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafabfff700 0 client.25793 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6702c30 0x5617c670ea00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1931065355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6703550 0x5617c670bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 >> 172.21.15.32:0/1931065355 conn(0x5617c662b8f0 msgr2=0x5617c67087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 shutdown_connections 2023-08-11T15:51:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 1 -- 172.21.15.32:0/1931065355 wait complete. 2023-08-11T15:51:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.376+0000 7fafe6680b80 20 client.25793 trim_cache size 0 max 16384 2023-08-11T15:51:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.394+0000 7fafe6680b80 1 Processor -- start 2023-08-11T15:51:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafe6680b80 1 -- start start 2023-08-11T15:51:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6702c30 0x5617c670e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c670b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c670ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafe6680b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c670d300 con 0x5617c6707320 2023-08-11T15:51:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafe6680b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c67233a0 con 0x5617c6703550 2023-08-11T15:51:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafe6680b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c66f1780 con 0x5617c6702c30 2023-08-11T15:51:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafd0b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6702c30 0x5617c670e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafcbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c670b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafd0b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6702c30 0x5617c670e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43174/0 (socket says 172.21.15.32:43174) 2023-08-11T15:51:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1025692424 learned_addr learned my addr 172.21.15.32:0/1025692424 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.395+0000 7fafd139c700 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c670ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1025692424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 msgr2=0x5617c670b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c670b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1025692424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 msgr2=0x5617c670ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c670ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafd0b9b700 1 -- 172.21.15.32:0/1025692424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c6465c00 con 0x5617c6702c30 2023-08-11T15:51:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6702c30 0x5617c670e530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fafc0000c00 tx=0x7fafc0009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2371aee16b84661 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafb77fe700 1 -- 172.21.15.32:0/1025692424 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0012db0 con 0x5617c6702c30 2023-08-11T15:51:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafb77fe700 1 -- 172.21.15.32:0/1025692424 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc0012820 con 0x5617c6702c30 2023-08-11T15:51:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.396+0000 7fafb77fe700 1 -- 172.21.15.32:0/1025692424 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00042a0 con 0x5617c6702c30 2023-08-11T15:51:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 -- 172.21.15.32:0/1025692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6702c30 msgr2=0x5617c670e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6702c30 0x5617c670e530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fafc0000c00 tx=0x7fafc0009fd0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 -- 172.21.15.32:0/1025692424 shutdown_connections 2023-08-11T15:51:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6707320 0x5617c670ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6702c30 0x5617c670e530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 --2- 172.21.15.32:0/1025692424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c670b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 -- 172.21.15.32:0/1025692424 >> 172.21.15.32:0/1025692424 conn(0x5617c662b8f0 msgr2=0x5617c6705370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 -- 172.21.15.32:0/1025692424 shutdown_connections 2023-08-11T15:51:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.397+0000 7fafe6680b80 1 -- 172.21.15.32:0/1025692424 wait complete. 2023-08-11T15:51:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.398+0000 7fafe6680b80 1 -- start start 2023-08-11T15:51:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.398+0000 7fafe6680b80 1 Processor -- start 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6702c30 0x5617c6711490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c67119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafe6680b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c67129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafe6680b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c6712f00 con 0x5617c6702c30 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafe6680b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c6713070 con 0x5617c6703550 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafe6680b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c6713230 con 0x5617c6707320 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafe6680b80 10 client.0 _ll_register_callbacks cb 0x7fafc400cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafcbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c67119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafcbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c67119d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55540/0 (socket says 172.21.15.32:55540) 2023-08-11T15:51:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafcbfff700 1 -- 172.21.15.32:0/473063549 learned_addr learned my addr 172.21.15.32:0/473063549 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafd0b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6702c30 0x5617c6711490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.399+0000 7fafd139c700 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c67129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafcbfff700 1 -- 172.21.15.32:0/473063549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 msgr2=0x5617c67129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafcbfff700 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c67129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafcbfff700 1 -- 172.21.15.32:0/473063549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6702c30 msgr2=0x5617c6711490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafcbfff700 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6702c30 0x5617c6711490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafcbfff700 1 -- 172.21.15.32:0/473063549 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617c647cd80 con 0x5617c6703550 2023-08-11T15:51:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafcbfff700 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c67119d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009810 tx=0x7fafb8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a08cc51c71acb20 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafabfff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:51:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafabfff700 1 -- 172.21.15.32:0/473063549 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb802e040 con 0x5617c6703550 2023-08-11T15:51:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.400+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617c66497c0 con 0x5617c6703550 2023-08-11T15:51:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafabfff700 1 -- 172.21.15.32:0/473063549 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fafb8034b70 con 0x5617c6703550 2023-08-11T15:51:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafabfff700 1 -- 172.21.15.32:0/473063549 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb8034d30 con 0x5617c6703550 2023-08-11T15:51:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 10 client.25799 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5617c64a31f0 con 0x5617c6703550 2023-08-11T15:51:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafabfff700 1 -- 172.21.15.32:0/473063549 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617c66497c0 con 0x5617c6703550 2023-08-11T15:51:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafabfff700 1 -- 172.21.15.32:0/473063549 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fafb800ba10 con 0x5617c6703550 2023-08-11T15:51:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 20 client.25799 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c67754d0 0x5617c67778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 4 client.25799 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:51:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.401+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617c6609ff0 con 0x5617c67754d0 2023-08-11T15:51:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.402+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c67754d0 0x5617c67778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.402+0000 7fafd0b9b700 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c67754d0 0x5617c67778f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252fdeccbed9b60e server_cookie=ebd698ef7c4e87c7 in_seq=0 out_seq=0 2023-08-11T15:51:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.402+0000 7fafabfff700 10 client.25799 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.653+0000 7fafabfff700 1 -- 172.21.15.32:0/473063549 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617c6609ff0 con 0x5617c67754d0 2023-08-11T15:51:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.653+0000 7fafabfff700 10 client.25799 handle_command_reply: tid=0 2023-08-11T15:51:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.653+0000 7fafe6680b80 1 client.25799 shutdown 2023-08-11T15:51:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.654+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 msgr2=0x5617c67119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.654+0000 7fafe6680b80 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c67119d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009810 tx=0x7fafb8009c00 comp rx=0 tx=0).stop 2023-08-11T15:51:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.655+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 shutdown_connections 2023-08-11T15:51:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.655+0000 7fafe6680b80 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x5617c67754d0 0x5617c67778f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.655+0000 7fafe6680b80 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5617c6702c30 0x5617c6711490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.655+0000 7fafe6680b80 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617c6707320 0x5617c67129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.655+0000 7fafe6680b80 1 --2- 172.21.15.32:0/473063549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617c6703550 0x5617c67119d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.655+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 >> 172.21.15.32:0/473063549 conn(0x5617c662b8f0 msgr2=0x5617c66f84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.655+0000 7fafabfff700 0 client.25799 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.656+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 shutdown_connections 2023-08-11T15:51:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.656+0000 7fafe6680b80 1 -- 172.21.15.32:0/473063549 wait complete. 2023-08-11T15:51:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.656+0000 7fafe6680b80 20 client.25799 trim_cache size 0 max 16384 2023-08-11T15:51:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:51:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:51:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:51:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:51:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fafbc0a00c0 msgr2=0x7fafbc0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fafbc0a00c0 0x7fafbc0a2580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fafb80098c0 tx=0x7fafb8005dc0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 msgr2=0x7fafcc196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 0x7fafcc196880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fafc400ef30 tx=0x7fafc400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 shutdown_connections 2023-08-11T15:51:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fafbc0a00c0 0x7fafbc0a2580 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafcc1068c0 0x7fafcc196880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafcc1008b0 0x7fafcc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 --2- 172.21.15.32:0/3545597424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafcc101270 0x7fafcc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.657+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 >> 172.21.15.32:0/3545597424 conn(0x7fafcc0fc380 msgr2=0x7fafcc0fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.658+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 shutdown_connections 2023-08-11T15:51:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:39.658+0000 7fafd2dff700 1 -- 172.21.15.32:0/3545597424 wait complete. 2023-08-11T15:51:39.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 555084 12.27 MB/sec execute 115 sec latency 158.089 ms 2023-08-11T15:51:40.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:40 smithi032 ceph-mon[97103]: pgmap v155: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2023-08-11T15:51:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:40 smithi192 ceph-mon[130806]: pgmap v155: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2023-08-11T15:51:40.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:40 smithi134 ceph-mon[130447]: pgmap v155: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2023-08-11T15:51:40.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 557949 12.32 MB/sec execute 116 sec latency 12.086 ms 2023-08-11T15:51:41.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 558956 12.23 MB/sec execute 117 sec latency 155.713 ms 2023-08-11T15:51:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:42 smithi192 ceph-mon[130806]: pgmap v156: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-08-11T15:51:42.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:42 smithi134 ceph-mon[130447]: pgmap v156: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-08-11T15:51:42.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 561311 12.26 MB/sec execute 118 sec latency 36.453 ms 2023-08-11T15:51:42.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:42 smithi032 ceph-mon[97103]: pgmap v156: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-08-11T15:51:43.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 563029 12.21 MB/sec execute 119 sec latency 67.235 ms 2023-08-11T15:51:44.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:44 smithi192 ceph-mon[130806]: pgmap v157: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 540 op/s 2023-08-11T15:51:44.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:44 smithi134 ceph-mon[130447]: pgmap v157: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 540 op/s 2023-08-11T15:51:44.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 565459 12.23 MB/sec execute 120 sec latency 13.639 ms 2023-08-11T15:51:44.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:44 smithi032 ceph-mon[97103]: pgmap v157: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 540 op/s 2023-08-11T15:51:45.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 567794 12.23 MB/sec execute 121 sec latency 41.124 ms 2023-08-11T15:51:46.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:46 smithi192 ceph-mon[130806]: pgmap v158: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.4 MiB/s wr, 469 op/s 2023-08-11T15:51:46.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:46 smithi134 ceph-mon[130447]: pgmap v158: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.4 MiB/s wr, 469 op/s 2023-08-11T15:51:46.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 569868 12.23 MB/sec execute 122 sec latency 15.199 ms 2023-08-11T15:51:46.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:46 smithi032 ceph-mon[97103]: pgmap v158: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.4 MiB/s wr, 469 op/s 2023-08-11T15:51:47.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 572303 12.24 MB/sec execute 123 sec latency 16.525 ms 2023-08-11T15:51:48.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:47 smithi134 ceph-mon[130447]: pgmap v159: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.0 MiB/s wr, 575 op/s 2023-08-11T15:51:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:47 smithi032 ceph-mon[97103]: pgmap v159: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.0 MiB/s wr, 575 op/s 2023-08-11T15:51:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:47 smithi192 ceph-mon[130806]: pgmap v159: 129 pgs: 129 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.0 MiB/s wr, 575 op/s 2023-08-11T15:51:48.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 574483 12.21 MB/sec execute 124 sec latency 19.789 ms 2023-08-11T15:51:49.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 576413 12.20 MB/sec execute 125 sec latency 33.519 ms 2023-08-11T15:51:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:50 smithi192 ceph-mon[130806]: pgmap v160: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 514 op/s 2023-08-11T15:51:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:50 smithi134 ceph-mon[130447]: pgmap v160: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 514 op/s 2023-08-11T15:51:50.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 579089 12.24 MB/sec execute 126 sec latency 97.319 ms 2023-08-11T15:51:50.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:50 smithi032 ceph-mon[97103]: pgmap v160: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 514 op/s 2023-08-11T15:51:51.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 580412 12.20 MB/sec execute 127 sec latency 169.091 ms 2023-08-11T15:51:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:52 smithi192 ceph-mon[130806]: pgmap v161: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 462 op/s 2023-08-11T15:51:52.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:52 smithi134 ceph-mon[130447]: pgmap v161: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 462 op/s 2023-08-11T15:51:52.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 582885 12.21 MB/sec execute 128 sec latency 15.377 ms 2023-08-11T15:51:52.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:52 smithi032 ceph-mon[97103]: pgmap v161: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 462 op/s 2023-08-11T15:51:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 -- 172.21.15.32:0/1999860890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 msgr2=0x7f9da0106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1999860890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 0x7f9da0106ca0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009990 tx=0x7f9d94009ca0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 -- 172.21.15.32:0/1999860890 shutdown_connections 2023-08-11T15:51:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1999860890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da0101270 0x7f9da010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1999860890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 0x7f9da0106ca0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1999860890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 0x7f9da0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 -- 172.21.15.32:0/1999860890 >> 172.21.15.32:0/1999860890 conn(0x7f9da00fc380 msgr2=0x7f9da00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 -- 172.21.15.32:0/1999860890 shutdown_connections 2023-08-11T15:51:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.817+0000 7f9da6dff700 1 -- 172.21.15.32:0/1999860890 wait complete. 2023-08-11T15:51:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.818+0000 7f9da6dff700 1 Processor -- start 2023-08-11T15:51:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.818+0000 7f9da6dff700 1 -- start start 2023-08-11T15:51:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da6dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 0x7f9da019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da6dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da0101270 0x7f9da019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da6dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 0x7f9da01967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da6dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da0196e30 con 0x7f9da0101270 2023-08-11T15:51:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da6dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da0196fa0 con 0x7f9da01068c0 2023-08-11T15:51:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da6dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da0197160 con 0x7f9da01008b0 2023-08-11T15:51:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da4b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 0x7f9da019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da539c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 0x7f9da01967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da4b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 0x7f9da019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:33842/0 (socket says 172.21.15.32:33842) 2023-08-11T15:51:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da539c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 0x7f9da01967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:46326/0 (socket says 172.21.15.32:46326) 2023-08-11T15:51:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9da4b9b700 1 -- 172.21.15.32:0/1192328876 learned_addr learned my addr 172.21.15.32:0/1192328876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.819+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da0101270 0x7f9da019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da4b9b700 1 -- 172.21.15.32:0/1192328876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 msgr2=0x7f9da01967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 0x7f9da01967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da4b9b700 1 -- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da0101270 msgr2=0x7f9da019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da0101270 0x7f9da019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da4b9b700 1 -- 172.21.15.32:0/1192328876 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d940095f0 con 0x7f9da01008b0 2023-08-11T15:51:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 0x7f9da019c960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9d94000c00 tx=0x7f9d94005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9d9dffb700 1 -- 172.21.15.32:0/1192328876 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9401d070 con 0x7f9da01008b0 2023-08-11T15:51:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9d9dffb700 1 -- 172.21.15.32:0/1192328876 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d94022470 con 0x7f9da01008b0 2023-08-11T15:51:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da01973e0 con 0x7f9da01008b0 2023-08-11T15:51:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9d9dffb700 1 -- 172.21.15.32:0/1192328876 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94017650 con 0x7f9da01008b0 2023-08-11T15:51:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.820+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da01a3950 con 0x7f9da01008b0 2023-08-11T15:51:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.824+0000 7f9d9dffb700 1 -- 172.21.15.32:0/1192328876 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9d940225e0 con 0x7f9da01008b0 2023-08-11T15:51:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 Processor -- start 2023-08-11T15:51:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 -- start start 2023-08-11T15:51:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119591260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958b630 0x55d11958bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958bff0 0x55d119597c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1194e45d0 con 0x55d11958b630 2023-08-11T15:51:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1194e1ef0 con 0x55d119590e80 2023-08-11T15:51:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.825+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1194dd7e0 con 0x55d11958bff0 2023-08-11T15:51:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9da539c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119591260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9da539c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119591260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:46332/0 (socket says 172.21.15.32:46332) 2023-08-11T15:51:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9da539c700 1 -- 172.21.15.32:0/1586609740 learned_addr learned my addr 172.21.15.32:0/1586609740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9dffb700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9d900a00c0 0x7f9d900a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958b630 0x55d11958bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958bff0 0x55d119597c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9ffff700 1 -- 172.21.15.32:0/1586609740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 msgr2=0x55d119591260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119591260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9ffff700 1 -- 172.21.15.32:0/1586609740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958b630 msgr2=0x55d11958bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958b630 0x55d11958bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9ffff700 1 -- 172.21.15.32:0/1586609740 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11935d5d0 con 0x55d11958bff0 2023-08-11T15:51:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9da539c700 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119591260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:51:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9d9dffb700 1 -- 172.21.15.32:0/1192328876 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9d940ccd70 con 0x7f9da01008b0 2023-08-11T15:51:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.826+0000 7f9da539c700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9d900a00c0 0x7f9d900a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.827+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958bff0 0x55d119597c10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009c80 tx=0x7f9d8c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef0fd471b44d44a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.827+0000 7f9d8affd700 1 -- 172.21.15.32:0/1586609740 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c011cb0 con 0x55d11958bff0 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.827+0000 7f9d8affd700 1 -- 172.21.15.32:0/1586609740 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d8c011e10 con 0x55d11958bff0 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.827+0000 7f9d8affd700 1 -- 172.21.15.32:0/1586609740 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0193e0 con 0x55d11958bff0 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1586609740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958bff0 msgr2=0x55d119597c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9da539c700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9d900a00c0 0x7f9d900a2580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d980076a0 tx=0x7f9d98009040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958bff0 0x55d119597c10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009c80 tx=0x7f9d8c009f90 comp rx=0 tx=0).stop 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1586609740 shutdown_connections 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958bff0 0x55d119597c10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119591260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1586609740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958b630 0x55d11958bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1586609740 >> 172.21.15.32:0/1586609740 conn(0x55d1194b88f0 msgr2=0x55d119585e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1586609740 shutdown_connections 2023-08-11T15:51:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.828+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1586609740 wait complete. 2023-08-11T15:51:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 -- start start 2023-08-11T15:51:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 Processor -- start 2023-08-11T15:51:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d11959a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d11959af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119594d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11947ba60 con 0x55d11958bff0 2023-08-11T15:51:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11947b760 con 0x55d119590e80 2023-08-11T15:51:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11935f8c0 con 0x55d11958b630 2023-08-11T15:51:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.829+0000 7f9dba7e1b80 10 client.0 _ll_register_callbacks cb 0x55d11947ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d11959a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9d9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d11959af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d11959a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:33872/0 (socket says 172.21.15.32:33872) 2023-08-11T15:51:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3127591039 learned_addr learned my addr 172.21.15.32:0/3127591039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da539c700 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119594d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 msgr2=0x55d119594d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119594d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3127591039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 msgr2=0x55d11959af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d11959af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3127591039 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d98011300 con 0x55d11958b630 2023-08-11T15:51:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d11959a9d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d94022890 tx=0x7f9d940ca6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=250fcda4cccd485d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9d7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:51:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.830+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3127591039 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9409a600 con 0x55d11958b630 2023-08-11T15:51:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d11957e7e0 con 0x55d11958b630 2023-08-11T15:51:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3127591039 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9d9409a8a0 con 0x55d11958b630 2023-08-11T15:51:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3127591039 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d94099f70 con 0x55d11958b630 2023-08-11T15:51:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 10 client.25813 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9d8c0098e0 con 0x55d11958b630 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3127591039 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d11957e7e0 con 0x55d11958b630 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3127591039 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d94099690 con 0x55d11958b630 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 10 client.25813 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 20 client.25813 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195a4bc0 0x55d1195a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 4 client.25813 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:51:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.831+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d11955a480 con 0x55d1195a4bc0 2023-08-11T15:51:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.832+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195a4bc0 0x55d1195a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.832+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195a4bc0 0x55d1195a6fb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c3cfa2a9dae4de server_cookie=c102a1a6a37e4419 in_seq=0 out_seq=0 2023-08-11T15:51:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.832+0000 7f9d7ffff700 10 client.25813 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3127591039 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d11955a480 con 0x55d1195a4bc0 2023-08-11T15:51:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9d7ffff700 10 client.25813 handle_command_reply: tid=0 2023-08-11T15:51:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 client.25813 shutdown 2023-08-11T15:51:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 msgr2=0x55d11959a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d11959a9d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d94022890 tx=0x7f9d940ca6b0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 shutdown_connections 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195a4bc0 0x55d1195a6fb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d119590e80 0x55d119594d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d11959a9d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9d7ffff700 0 client.25813 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3127591039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d11959af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 >> 172.21.15.32:0/3127591039 conn(0x55d1194b88f0 msgr2=0x55d1195855d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 shutdown_connections 2023-08-11T15:51:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.833+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3127591039 wait complete. 2023-08-11T15:51:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.834+0000 7f9dba7e1b80 20 client.25813 trim_cache size 0 max 16384 2023-08-11T15:51:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.850+0000 7f9dba7e1b80 1 Processor -- start 2023-08-11T15:51:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9dba7e1b80 1 -- start start 2023-08-11T15:51:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958b630 0x55d11959a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d119585d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d119590e80 0x55d11958fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1195877b0 con 0x55d11958bff0 2023-08-11T15:51:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d119586290 con 0x55d11958b630 2023-08-11T15:51:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d119586400 con 0x55d119590e80 2023-08-11T15:51:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9da4b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958b630 0x55d11959a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9da4b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958b630 0x55d11959a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:46344/0 (socket says 172.21.15.32:46344) 2023-08-11T15:51:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3019867081 learned_addr learned my addr 172.21.15.32:0/3019867081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d119585d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.851+0000 7f9da539c700 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d119590e80 0x55d11958fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3019867081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d119590e80 msgr2=0x55d11958fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d119590e80 0x55d11958fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3019867081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 msgr2=0x55d119585d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d119585d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9da4b9b700 1 -- 172.21.15.32:0/3019867081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d119309d80 con 0x55d11958b630 2023-08-11T15:51:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958b630 0x55d11959a530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d9409e170 tx=0x7f9d940093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1d4b0313de4d3b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9d8b7fe700 1 -- 172.21.15.32:0/3019867081 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94099f70 con 0x55d11958b630 2023-08-11T15:51:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9d8b7fe700 1 -- 172.21.15.32:0/3019867081 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d940a1f20 con 0x55d11958b630 2023-08-11T15:51:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.852+0000 7f9d8b7fe700 1 -- 172.21.15.32:0/3019867081 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9409a230 con 0x55d11958b630 2023-08-11T15:51:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3019867081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958b630 msgr2=0x55d11959a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958b630 0x55d11959a530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d9409e170 tx=0x7f9d940093b0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3019867081 shutdown_connections 2023-08-11T15:51:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d119590e80 0x55d11958fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958b630 0x55d11959a530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/3019867081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11958bff0 0x55d119585d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3019867081 >> 172.21.15.32:0/3019867081 conn(0x55d1194b88f0 msgr2=0x55d11958edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3019867081 shutdown_connections 2023-08-11T15:51:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/3019867081 wait complete. 2023-08-11T15:51:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.853+0000 7f9dba7e1b80 1 -- start start 2023-08-11T15:51:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 1 Processor -- start 2023-08-11T15:51:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d1195f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958bff0 0x55d1195f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d119590e80 0x55d1195fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1195fc140 con 0x55d119590e80 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1195fc280 con 0x55d11958bff0 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1195fc440 con 0x55d11958b630 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9dba7e1b80 10 client.0 _ll_register_callbacks cb 0x55d11947ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9d9ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958bff0 0x55d1195f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9d9ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958bff0 0x55d1195f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:46354/0 (socket says 172.21.15.32:46354) 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9d9ffff700 1 -- 172.21.15.32:0/1412863077 learned_addr learned my addr 172.21.15.32:0/1412863077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d1195f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.854+0000 7f9da539c700 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d119590e80 0x55d1195fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d9ffff700 1 -- 172.21.15.32:0/1412863077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 msgr2=0x55d1195f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d1195f0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d9ffff700 1 -- 172.21.15.32:0/1412863077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d119590e80 msgr2=0x55d1195fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d119590e80 0x55d1195fbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d9ffff700 1 -- 172.21.15.32:0/1412863077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c0098e0 con 0x55d11958bff0 2023-08-11T15:51:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d9ffff700 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958bff0 0x55d1195f12b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d119586a30 tx=0x7f9d8c0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27bac4a48e9db11a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:51:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:51:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d7ffff700 1 -- 172.21.15.32:0/1412863077 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c02c070 con 0x55d11958bff0 2023-08-11T15:51:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d8c011e10 con 0x55d11958bff0 2023-08-11T15:51:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.855+0000 7f9d7ffff700 1 -- 172.21.15.32:0/1412863077 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9d8c011bf0 con 0x55d11958bff0 2023-08-11T15:51:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9d7ffff700 1 -- 172.21.15.32:0/1412863077 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d8c005040 con 0x55d11958bff0 2023-08-11T15:51:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 10 client.25817 fetch_fsmap learned FSMap version 27 2023-08-11T15:51:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d1192f2c00 con 0x55d11958bff0 2023-08-11T15:51:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9d7ffff700 1 -- 172.21.15.32:0/1412863077 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d8c03a080 con 0x55d11958bff0 2023-08-11T15:51:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9d7ffff700 1 -- 172.21.15.32:0/1412863077 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d8c01a440 con 0x55d11958bff0 2023-08-11T15:51:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:51:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:51:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 20 client.25817 populate_metadata read hostname 'smithi032' 2023-08-11T15:51:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195fdc20 0x55d119600040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:51:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 4 client.25817 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:51:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d1194d67c0 con 0x55d1195fdc20 2023-08-11T15:51:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.856+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195fdc20 0x55d119600040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:51:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.857+0000 7f9da4b9b700 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195fdc20 0x55d119600040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e39c8580e3afcce server_cookie=1349b4064a6db2d1 in_seq=0 out_seq=0 2023-08-11T15:51:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:52.857+0000 7f9d7ffff700 10 client.25817 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.161+0000 7f9d7ffff700 1 -- 172.21.15.32:0/1412863077 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d1194d67c0 con 0x55d1195fdc20 2023-08-11T15:51:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.161+0000 7f9d7ffff700 10 client.25817 handle_command_reply: tid=0 2023-08-11T15:51:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.161+0000 7f9dba7e1b80 1 client.25817 shutdown 2023-08-11T15:51:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958bff0 msgr2=0x55d1195f12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958bff0 0x55d1195f12b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d119586a30 tx=0x7f9d8c0096d0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 shutdown_connections 2023-08-11T15:51:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d1195fdc20 0x55d119600040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d119590e80 0x55d1195fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d11958b630 0x55d1195f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 --2- 172.21.15.32:0/1412863077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d11958bff0 0x55d1195f12b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9d7ffff700 0 client.25817 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:51:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.163+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 >> 172.21.15.32:0/1412863077 conn(0x55d1194b88f0 msgr2=0x55d11958fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.164+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 shutdown_connections 2023-08-11T15:51:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.164+0000 7f9dba7e1b80 1 -- 172.21.15.32:0/1412863077 wait complete. 2023-08-11T15:51:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.164+0000 7f9dba7e1b80 20 client.25817 trim_cache size 0 max 16384 2023-08-11T15:51:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:51:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:51:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:51:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:51:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9d900a00c0 msgr2=0x7f9d900a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9d900a00c0 0x7f9d900a2580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d980076a0 tx=0x7f9d98009040 comp rx=0 tx=0).stop 2023-08-11T15:51:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 msgr2=0x7f9da019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:51:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 0x7f9da019c960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9d94000c00 tx=0x7f9d94005740 comp rx=0 tx=0).stop 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 shutdown_connections 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9d900a00c0 0x7f9d900a2580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9da01068c0 0x7f9da01967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da01008b0 0x7f9da019c960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 --2- 172.21.15.32:0/1192328876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da0101270 0x7f9da019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 >> 172.21.15.32:0/1192328876 conn(0x7f9da00fc380 msgr2=0x7f9da00fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 shutdown_connections 2023-08-11T15:51:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:51:53.165+0000 7f9da6dff700 1 -- 172.21.15.32:0/1192328876 wait complete. 2023-08-11T15:51:53.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 584198 12.16 MB/sec execute 129 sec latency 158.245 ms 2023-08-11T15:51:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:54 smithi192 ceph-mon[130806]: pgmap v162: 129 pgs: 129 active+clean; 71 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2023-08-11T15:51:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:54 smithi134 ceph-mon[130447]: pgmap v162: 129 pgs: 129 active+clean; 71 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2023-08-11T15:51:54.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 586482 12.17 MB/sec execute 130 sec latency 31.627 ms 2023-08-11T15:51:54.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:54 smithi032 ceph-mon[97103]: pgmap v162: 129 pgs: 129 active+clean; 71 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2023-08-11T15:51:55.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:55 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:51:55.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:55 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:51:55.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 588418 12.13 MB/sec execute 131 sec latency 26.829 ms 2023-08-11T15:51:55.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:55 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:51:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:56 smithi192 ceph-mon[130806]: pgmap v163: 129 pgs: 129 active+clean; 71 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 187 op/s 2023-08-11T15:51:56.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:56 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:51:56.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:56 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:51:56.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:56 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:51:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:56 smithi134 ceph-mon[130447]: pgmap v163: 129 pgs: 129 active+clean; 71 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 187 op/s 2023-08-11T15:51:56.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:56 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:51:56.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:56 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:51:56.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:56 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:51:56.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 590214 12.13 MB/sec execute 132 sec latency 75.315 ms 2023-08-11T15:51:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:56 smithi032 ceph-mon[97103]: pgmap v163: 129 pgs: 129 active+clean; 71 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 187 op/s 2023-08-11T15:51:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:56 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:51:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:56 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:51:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:56 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:51:57.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 592807 12.14 MB/sec execute 133 sec latency 12.569 ms 2023-08-11T15:51:58.220 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:51:57 smithi134 ceph-mon[130447]: pgmap v164: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 373 op/s 2023-08-11T15:51:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:51:57 smithi032 ceph-mon[97103]: pgmap v164: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 373 op/s 2023-08-11T15:51:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:51:57 smithi192 ceph-mon[130806]: pgmap v164: 129 pgs: 129 active+clean; 64 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 373 op/s 2023-08-11T15:51:58.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 594612 12.13 MB/sec execute 134 sec latency 32.685 ms 2023-08-11T15:51:59.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 597079 12.14 MB/sec execute 135 sec latency 12.918 ms 2023-08-11T15:52:00.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:00 smithi032 ceph-mon[97103]: pgmap v165: 129 pgs: 129 active+clean; 61 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 310 op/s 2023-08-11T15:52:00.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:00 smithi192 ceph-mon[130806]: pgmap v165: 129 pgs: 129 active+clean; 61 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 310 op/s 2023-08-11T15:52:00.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:00 smithi134 ceph-mon[130447]: pgmap v165: 129 pgs: 129 active+clean; 61 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 310 op/s 2023-08-11T15:52:00.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 598982 12.10 MB/sec execute 136 sec latency 58.311 ms 2023-08-11T15:52:01.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 600081 12.07 MB/sec execute 137 sec latency 79.330 ms 2023-08-11T15:52:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:02 smithi032 ceph-mon[97103]: pgmap v166: 129 pgs: 129 active+clean; 61 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-08-11T15:52:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:02 smithi192 ceph-mon[130806]: pgmap v166: 129 pgs: 129 active+clean; 61 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-08-11T15:52:02.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:02 smithi134 ceph-mon[130447]: pgmap v166: 129 pgs: 129 active+clean; 61 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-08-11T15:52:02.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 601950 12.06 MB/sec execute 138 sec latency 15.924 ms 2023-08-11T15:52:03.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 604369 12.07 MB/sec execute 139 sec latency 49.194 ms 2023-08-11T15:52:04.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:04 smithi032 ceph-mon[97103]: pgmap v167: 129 pgs: 129 active+clean; 67 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 388 op/s 2023-08-11T15:52:04.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:04 smithi192 ceph-mon[130806]: pgmap v167: 129 pgs: 129 active+clean; 67 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 388 op/s 2023-08-11T15:52:04.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:04 smithi134 ceph-mon[130447]: pgmap v167: 129 pgs: 129 active+clean; 67 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 388 op/s 2023-08-11T15:52:04.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 606824 12.07 MB/sec execute 140 sec latency 14.770 ms 2023-08-11T15:52:05.758 INFO:tasks.workunit.client.0.smithi032.stdout: 1 608770 12.07 MB/sec execute 141 sec latency 12.146 ms 2023-08-11T15:52:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.341+0000 7fcb003ff700 1 -- 172.21.15.32:0/1990259667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf80686d0 msgr2=0x7fcaf810d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.341+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1990259667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf80686d0 0x7fcaf810d720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000b0d0 tx=0x7fcaf000b3e0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.342+0000 7fcb003ff700 1 -- 172.21.15.32:0/1990259667 shutdown_connections 2023-08-11T15:52:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.342+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1990259667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf80686d0 0x7fcaf810d720 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.342+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1990259667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcaf81068d0 0x7fcaf8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.342+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1990259667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf8067d10 0x7fcaf8068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.342+0000 7fcb003ff700 1 -- 172.21.15.32:0/1990259667 >> 172.21.15.32:0/1990259667 conn(0x7fcaf8075250 msgr2=0x7fcaf8075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.342+0000 7fcb003ff700 1 -- 172.21.15.32:0/1990259667 shutdown_connections 2023-08-11T15:52:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.342+0000 7fcb003ff700 1 -- 172.21.15.32:0/1990259667 wait complete. 2023-08-11T15:52:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.343+0000 7fcb003ff700 1 Processor -- start 2023-08-11T15:52:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.343+0000 7fcb003ff700 1 -- start start 2023-08-11T15:52:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.343+0000 7fcb003ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf8067d10 0x7fcaf819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.343+0000 7fcb003ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf80686d0 0x7fcaf819d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcb003ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcaf81068d0 0x7fcaf8196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcb003ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8197060 con 0x7fcaf8067d10 2023-08-11T15:52:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcb003ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81971d0 con 0x7fcaf80686d0 2023-08-11T15:52:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcb003ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8197390 con 0x7fcaf81068d0 2023-08-11T15:52:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf80686d0 0x7fcaf819d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafe19b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf8067d10 0x7fcaf819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf80686d0 0x7fcaf819d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50388/0 (socket says 172.21.15.32:50388) 2023-08-11T15:52:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 -- 172.21.15.32:0/1435589763 learned_addr learned my addr 172.21.15.32:0/1435589763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafe99c700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcaf81068d0 0x7fcaf8196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 -- 172.21.15.32:0/1435589763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcaf81068d0 msgr2=0x7fcaf8196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcaf81068d0 0x7fcaf8196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 -- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf8067d10 msgr2=0x7fcaf819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf8067d10 0x7fcaf819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafd99a700 1 -- 172.21.15.32:0/1435589763 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0009d00 con 0x7fcaf80686d0 2023-08-11T15:52:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafe19b700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf8067d10 0x7fcaf819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.344+0000 7fcafe99c700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcaf81068d0 0x7fcaf8196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.345+0000 7fcafd99a700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf80686d0 0x7fcaf819d0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9a0 tx=0x7fcae800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.345+0000 7fcaef7fe700 1 -- 172.21.15.32:0/1435589763 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8009790 con 0x7fcaf80686d0 2023-08-11T15:52:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.345+0000 7fcaef7fe700 1 -- 172.21.15.32:0/1435589763 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcae800cbd0 con 0x7fcaf80686d0 2023-08-11T15:52:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.345+0000 7fcaef7fe700 1 -- 172.21.15.32:0/1435589763 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8018680 con 0x7fcaf80686d0 2023-08-11T15:52:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.345+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf8197670 con 0x7fcaf80686d0 2023-08-11T15:52:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.345+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf80ffc60 con 0x7fcaf80686d0 2023-08-11T15:52:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.349+0000 7fcaef7fe700 1 -- 172.21.15.32:0/1435589763 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fcae800cd40 con 0x7fcaf80686d0 2023-08-11T15:52:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 Processor -- start 2023-08-11T15:52:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 -- start start 2023-08-11T15:52:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c8468ae80 0x559c8468b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685630 0x559c84685ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685ff0 0x559c84691c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c845de5d0 con 0x559c8468ae80 2023-08-11T15:52:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c845dbef0 con 0x559c84685630 2023-08-11T15:52:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c845d77e0 con 0x559c84685ff0 2023-08-11T15:52:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcaef7fe700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcae40a00c0 0x7fcae40a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcafe19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685630 0x559c84685ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcafe19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685630 0x559c84685ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50390/0 (socket says 172.21.15.32:50390) 2023-08-11T15:52:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.350+0000 7fcafe19b700 1 -- 172.21.15.32:0/2464686898 learned_addr learned my addr 172.21.15.32:0/2464686898 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafd99a700 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685ff0 0x559c84691c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcaef7fe700 1 -- 172.21.15.32:0/1435589763 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcae8014070 con 0x7fcaf80686d0 2023-08-11T15:52:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe19b700 1 -- 172.21.15.32:0/2464686898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685ff0 msgr2=0x559c84691c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe99c700 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c8468ae80 0x559c8468b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685ff0 0x559c84691c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe19b700 1 -- 172.21.15.32:0/2464686898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c8468ae80 msgr2=0x559c8468b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c8468ae80 0x559c8468b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe19b700 1 -- 172.21.15.32:0/2464686898 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c844575d0 con 0x559c84685630 2023-08-11T15:52:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe99c700 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c8468ae80 0x559c8468b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:52:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe99c700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcae40a00c0 0x7fcae40a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685630 0x559c84685ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40115f0 tx=0x7fcaf4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4c597a7ddef209 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.351+0000 7fcaecbfe700 1 -- 172.21.15.32:0/2464686898 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4019630 con 0x559c84685630 2023-08-11T15:52:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcaecbfe700 1 -- 172.21.15.32:0/2464686898 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf4019c50 con 0x559c84685630 2023-08-11T15:52:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcaecbfe700 1 -- 172.21.15.32:0/2464686898 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400ec10 con 0x559c84685630 2023-08-11T15:52:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2464686898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685630 msgr2=0x559c84685ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685630 0x559c84685ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40115f0 tx=0x7fcaf4011900 comp rx=0 tx=0).stop 2023-08-11T15:52:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcafe99c700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcae40a00c0 0x7fcae40a2580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000b770 tx=0x7fcaf00099e0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2464686898 shutdown_connections 2023-08-11T15:52:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c8468ae80 0x559c8468b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685ff0 0x559c84691c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2464686898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685630 0x559c84685ab0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2464686898 >> 172.21.15.32:0/2464686898 conn(0x559c845b28f0 msgr2=0x559c8467fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2464686898 shutdown_connections 2023-08-11T15:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.352+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2464686898 wait complete. 2023-08-11T15:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.353+0000 7fcb13ccbb80 1 -- start start 2023-08-11T15:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.353+0000 7fcb13ccbb80 1 Processor -- start 2023-08-11T15:52:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685630 0x559c84694a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685ff0 0x559c84694f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c8468ae80 0x559c8468edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c84575a60 con 0x559c84685630 2023-08-11T15:52:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c84575760 con 0x559c84685ff0 2023-08-11T15:52:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c844598c0 con 0x559c8468ae80 2023-08-11T15:52:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcb13ccbb80 10 client.0 _ll_register_callbacks cb 0x559c84578a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685ff0 0x559c84694f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafe19b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685630 0x559c84694a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafe99c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c8468ae80 0x559c8468edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685ff0 0x559c84694f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50394/0 (socket says 172.21.15.32:50394) 2023-08-11T15:52:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 -- 172.21.15.32:0/2875667799 learned_addr learned my addr 172.21.15.32:0/2875667799 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 -- 172.21.15.32:0/2875667799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c8468ae80 msgr2=0x559c8468edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c8468ae80 0x559c8468edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 -- 172.21.15.32:0/2875667799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685630 msgr2=0x559c84694a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685630 0x559c84694a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.354+0000 7fcafd99a700 1 -- 172.21.15.32:0/2875667799 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf4009d50 con 0x559c84685ff0 2023-08-11T15:52:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcafd99a700 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685ff0 0x559c84694f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcae8095510 tx=0x7fcae8007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799f12a90e89a4d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcae1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:52:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2875667799 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8098970 con 0x559c84685ff0 2023-08-11T15:52:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2875667799 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcae8099710 con 0x559c84685ff0 2023-08-11T15:52:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c846787e0 con 0x559c84685ff0 2023-08-11T15:52:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2875667799 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae8025170 con 0x559c84685ff0 2023-08-11T15:52:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcb13ccbb80 10 client.25829 fetch_fsmap learned FSMap version 27 2023-08-11T15:52:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559c84403d80 con 0x559c84685ff0 2023-08-11T15:52:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2875667799 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c846787e0 con 0x559c84685ff0 2023-08-11T15:52:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2875667799 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcae80a0990 con 0x559c84685ff0 2023-08-11T15:52:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcb13ccbb80 10 client.25829 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:52:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.355+0000 7fcb13ccbb80 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.356+0000 7fcb13ccbb80 20 client.25829 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.356+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c8469ef10 0x559c846a1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.356+0000 7fcb13ccbb80 4 client.25829 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.356+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c84654480 con 0x559c8469ef10 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.356+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c8469ef10 0x559c846a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.356+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c8469ef10 0x559c846a1330 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303d4c14cfa771da server_cookie=3e02425d949e0c14 in_seq=0 out_seq=0 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.356+0000 7fcae1ffb700 10 client.25829 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2875667799 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559c84654480 con 0x559c8469ef10 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcae1ffb700 10 client.25829 handle_command_reply: tid=0 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 client.25829 shutdown 2023-08-11T15:52:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685ff0 msgr2=0x559c84694f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685ff0 0x559c84694f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcae8095510 tx=0x7fcae8007ed0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 shutdown_connections 2023-08-11T15:52:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c8469ef10 0x559c846a1330 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685630 0x559c84694a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c8468ae80 0x559c8468edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2875667799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c84685ff0 0x559c84694f70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcae1ffb700 0 client.25829 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.357+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 >> 172.21.15.32:0/2875667799 conn(0x559c845b28f0 msgr2=0x559c8467f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.358+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 shutdown_connections 2023-08-11T15:52:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.358+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2875667799 wait complete. 2023-08-11T15:52:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.358+0000 7fcb13ccbb80 20 client.25829 trim_cache size 0 max 16384 2023-08-11T15:52:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.373+0000 7fcb13ccbb80 1 Processor -- start 2023-08-11T15:52:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.373+0000 7fcb13ccbb80 1 -- start start 2023-08-11T15:52:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c8467fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84688390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c84687860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c84681db0 con 0x559c84685ff0 2023-08-11T15:52:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c84680760 con 0x559c8468ae80 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c84678570 con 0x559c84685630 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe99c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c84687860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafd99a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84688390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe99c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c84687860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50396/0 (socket says 172.21.15.32:50396) 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe99c700 1 -- 172.21.15.32:0/3125012227 learned_addr learned my addr 172.21.15.32:0/3125012227 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe19b700 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c8467fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe99c700 1 -- 172.21.15.32:0/3125012227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 msgr2=0x559c8467fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe99c700 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c8467fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe99c700 1 -- 172.21.15.32:0/3125012227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 msgr2=0x559c84688390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.374+0000 7fcafe99c700 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84688390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.375+0000 7fcafe99c700 1 -- 172.21.15.32:0/3125012227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c843ecc00 con 0x559c8468ae80 2023-08-11T15:52:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.375+0000 7fcafe99c700 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c84687860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0007da0 tx=0x7fcaf001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1a41512506b53c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.375+0000 7fcaed3ff700 1 -- 172.21.15.32:0/3125012227 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000e040 con 0x559c8468ae80 2023-08-11T15:52:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.375+0000 7fcaed3ff700 1 -- 172.21.15.32:0/3125012227 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf0007dd0 con 0x559c8468ae80 2023-08-11T15:52:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.375+0000 7fcaed3ff700 1 -- 172.21.15.32:0/3125012227 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf002a670 con 0x559c8468ae80 2023-08-11T15:52:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/3125012227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 msgr2=0x559c84687860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c84687860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0007da0 tx=0x7fcaf001c000 comp rx=0 tx=0).stop 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/3125012227 shutdown_connections 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c84687860 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c8467fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/3125012227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84688390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/3125012227 >> 172.21.15.32:0/3125012227 conn(0x559c845b28f0 msgr2=0x559c8468f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/3125012227 shutdown_connections 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.376+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/3125012227 wait complete. 2023-08-11T15:52:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.377+0000 7fcb13ccbb80 1 -- start start 2023-08-11T15:52:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.377+0000 7fcb13ccbb80 1 Processor -- start 2023-08-11T15:52:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.377+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c846926d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.377+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84692c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcb13ccbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c846f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c846f2140 con 0x559c84685ff0 2023-08-11T15:52:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c846f2280 con 0x559c8468ae80 2023-08-11T15:52:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcb13ccbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c846f23c0 con 0x559c84685630 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcb13ccbb80 10 client.0 _ll_register_callbacks cb 0x7fcae8010b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c846926d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c846926d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:55340/0 (socket says 172.21.15.32:55340) 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafd99a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84692c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 -- 172.21.15.32:0/2436411843 learned_addr learned my addr 172.21.15.32:0/2436411843 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe99c700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c846f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 -- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 msgr2=0x559c846f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c846f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 -- 172.21.15.32:0/2436411843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 msgr2=0x559c84692c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84692c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe19b700 1 -- 172.21.15.32:0/2436411843 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf810e020 con 0x559c84685630 2023-08-11T15:52:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafd99a700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84692c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.378+0000 7fcafe99c700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c846f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcafe19b700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c846926d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4011510 tx=0x7fcaf40058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ea9c90c3ef6ead0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcae1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:52:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2436411843 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40052b0 con 0x559c84685630 2023-08-11T15:52:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaf4019c50 con 0x559c84685630 2023-08-11T15:52:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2436411843 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcaf4029070 con 0x559c84685630 2023-08-11T15:52:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2436411843 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf4005410 con 0x559c84685630 2023-08-11T15:52:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcb13ccbb80 10 client.25837 fetch_fsmap learned FSMap version 27 2023-08-11T15:52:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559c8442a1f0 con 0x559c84685630 2023-08-11T15:52:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2436411843 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf4019c50 con 0x559c84685630 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2436411843 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcaf40194c0 con 0x559c84685630 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.379+0000 7fcb13ccbb80 10 client.25837 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcb13ccbb80 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcb13ccbb80 20 client.25837 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c846f3a90 0x559c846f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcb13ccbb80 4 client.25837 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c845d07c0 con 0x559c846f3a90 2023-08-11T15:52:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcafd99a700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c846f3a90 0x559c846f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcafd99a700 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c846f3a90 0x559c846f5eb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f177d0cc9c7eca6f server_cookie=5545c342d917186f in_seq=0 out_seq=0 2023-08-11T15:52:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.380+0000 7fcae1ffb700 10 client.25837 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:06 smithi192 ceph-mon[130806]: pgmap v168: 129 pgs: 129 active+clean; 67 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 321 op/s 2023-08-11T15:52:06.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:06 smithi134 ceph-mon[130447]: pgmap v168: 129 pgs: 129 active+clean; 67 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 321 op/s 2023-08-11T15:52:06.759 INFO:tasks.workunit.client.0.smithi032.stdout: 1 609722 12.00 MB/sec execute 142 sec latency 256.640 ms 2023-08-11T15:52:06.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:06 smithi032 ceph-mon[97103]: pgmap v168: 129 pgs: 129 active+clean; 67 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 321 op/s 2023-08-11T15:52:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.973+0000 7fcae1ffb700 1 -- 172.21.15.32:0/2436411843 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c845d07c0 con 0x559c846f3a90 2023-08-11T15:52:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.973+0000 7fcae1ffb700 10 client.25837 handle_command_reply: tid=0 2023-08-11T15:52:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.973+0000 7fcb13ccbb80 1 client.25837 shutdown 2023-08-11T15:52:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 msgr2=0x559c846926d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c846926d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4011510 tx=0x7fcaf40058e0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 shutdown_connections 2023-08-11T15:52:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x559c846f3a90 0x559c846f5eb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c8468ae80 0x559c846f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c84685630 0x559c846926d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 --2- 172.21.15.32:0/2436411843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559c84685ff0 0x559c84692c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.974+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 >> 172.21.15.32:0/2436411843 conn(0x559c845b28f0 msgr2=0x559c8468f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.976+0000 7fcae1ffb700 0 client.25837 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.976+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 shutdown_connections 2023-08-11T15:52:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.976+0000 7fcb13ccbb80 1 -- 172.21.15.32:0/2436411843 wait complete. 2023-08-11T15:52:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:52:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:52:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:52:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:52:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.981+0000 7fcb13ccbb80 20 client.25837 trim_cache size 0 max 16384 2023-08-11T15:52:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.982+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcae40a00c0 msgr2=0x7fcae40a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.982+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcae40a00c0 0x7fcae40a2580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000b770 tx=0x7fcaf00099e0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.982+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf80686d0 msgr2=0x7fcaf819d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.982+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf80686d0 0x7fcaf819d0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9a0 tx=0x7fcae800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.984+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 shutdown_connections 2023-08-11T15:52:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.984+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcae40a00c0 0x7fcae40a2580 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.984+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaf8067d10 0x7fcaf819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.984+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcaf81068d0 0x7fcaf8196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.984+0000 7fcb003ff700 1 --2- 172.21.15.32:0/1435589763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaf80686d0 0x7fcaf819d0e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.984+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 >> 172.21.15.32:0/1435589763 conn(0x7fcaf8075250 msgr2=0x7fcaf80fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.985+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 shutdown_connections 2023-08-11T15:52:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:06.986+0000 7fcb003ff700 1 -- 172.21.15.32:0/1435589763 wait complete. 2023-08-11T15:52:07.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 609823 11.92 MB/sec execute 143 sec latency 306.618 ms 2023-08-11T15:52:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:07 smithi032 ceph-mon[97103]: pgmap v169: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 519 op/s 2023-08-11T15:52:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:07 smithi192 ceph-mon[130806]: pgmap v169: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 519 op/s 2023-08-11T15:52:08.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:07 smithi134 ceph-mon[130447]: pgmap v169: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 519 op/s 2023-08-11T15:52:08.759 INFO:tasks.workunit.client.0.smithi032.stdout: 1 611556 11.92 MB/sec execute 144 sec latency 31.582 ms 2023-08-11T15:52:09.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 614087 11.93 MB/sec execute 145 sec latency 12.855 ms 2023-08-11T15:52:10.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:10 smithi032 ceph-mon[97103]: pgmap v170: 129 pgs: 129 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2023-08-11T15:52:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:10 smithi192 ceph-mon[130806]: pgmap v170: 129 pgs: 129 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2023-08-11T15:52:10.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:10 smithi134 ceph-mon[130447]: pgmap v170: 129 pgs: 129 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 394 op/s 2023-08-11T15:52:10.759 INFO:tasks.workunit.client.0.smithi032.stdout: 1 615847 11.92 MB/sec execute 146 sec latency 27.933 ms 2023-08-11T15:52:11.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 618296 11.93 MB/sec execute 147 sec latency 15.902 ms 2023-08-11T15:52:12.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:12 smithi032 ceph-mon[97103]: pgmap v171: 129 pgs: 129 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.4 MiB/s wr, 351 op/s 2023-08-11T15:52:12.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:12 smithi192 ceph-mon[130806]: pgmap v171: 129 pgs: 129 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.4 MiB/s wr, 351 op/s 2023-08-11T15:52:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:12 smithi134 ceph-mon[130447]: pgmap v171: 129 pgs: 129 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.4 MiB/s wr, 351 op/s 2023-08-11T15:52:12.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 620124 11.89 MB/sec execute 148 sec latency 78.366 ms 2023-08-11T15:52:13.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 622244 11.90 MB/sec execute 149 sec latency 42.294 ms 2023-08-11T15:52:14.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:14 smithi192 ceph-mon[130806]: pgmap v172: 129 pgs: 129 active+clean; 63 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 567 op/s 2023-08-11T15:52:14.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:14 smithi134 ceph-mon[130447]: pgmap v172: 129 pgs: 129 active+clean; 63 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 567 op/s 2023-08-11T15:52:14.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 624882 11.92 MB/sec execute 150 sec latency 12.699 ms 2023-08-11T15:52:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:14 smithi032 ceph-mon[97103]: pgmap v172: 129 pgs: 129 active+clean; 63 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 567 op/s 2023-08-11T15:52:15.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 626606 11.90 MB/sec execute 151 sec latency 28.568 ms 2023-08-11T15:52:16.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:16 smithi192 ceph-mon[130806]: pgmap v173: 129 pgs: 129 active+clean; 63 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-08-11T15:52:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:16 smithi134 ceph-mon[130447]: pgmap v173: 129 pgs: 129 active+clean; 63 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-08-11T15:52:16.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 629089 11.92 MB/sec execute 152 sec latency 15.350 ms 2023-08-11T15:52:16.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:16 smithi032 ceph-mon[97103]: pgmap v173: 129 pgs: 129 active+clean; 63 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-08-11T15:52:17.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 631639 11.92 MB/sec execute 153 sec latency 14.551 ms 2023-08-11T15:52:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:17 smithi032 ceph-mon[97103]: pgmap v174: 129 pgs: 129 active+clean; 65 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 496 op/s 2023-08-11T15:52:18.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:17 smithi192 ceph-mon[130806]: pgmap v174: 129 pgs: 129 active+clean; 65 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 496 op/s 2023-08-11T15:52:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:17 smithi134 ceph-mon[130447]: pgmap v174: 129 pgs: 129 active+clean; 65 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 496 op/s 2023-08-11T15:52:18.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 633225 11.90 MB/sec execute 154 sec latency 63.616 ms 2023-08-11T15:52:19.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 635964 11.93 MB/sec execute 155 sec latency 11.834 ms 2023-08-11T15:52:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 -- 172.21.15.32:0/1486112777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 msgr2=0x7f70b0100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 --2- 172.21.15.32:0/1486112777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0100d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009990 tx=0x7f70a0009ca0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 -- 172.21.15.32:0/1486112777 shutdown_connections 2023-08-11T15:52:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 --2- 172.21.15.32:0/1486112777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b0101270 0x7f70b010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 --2- 172.21.15.32:0/1486112777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70b01068c0 0x7f70b0106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 --2- 172.21.15.32:0/1486112777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0100d30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 -- 172.21.15.32:0/1486112777 >> 172.21.15.32:0/1486112777 conn(0x7f70b00fc380 msgr2=0x7f70b00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 -- 172.21.15.32:0/1486112777 shutdown_connections 2023-08-11T15:52:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.180+0000 7f70b79ff700 1 -- 172.21.15.32:0/1486112777 wait complete. 2023-08-11T15:52:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.181+0000 7f70b79ff700 1 Processor -- start 2023-08-11T15:52:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b79ff700 1 -- start start 2023-08-11T15:52:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b79ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b79ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b0101270 0x7f70b0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b79ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70b01068c0 0x7f70b00736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b79ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b01a9430 con 0x7f70b0101270 2023-08-11T15:52:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b79ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b01a95a0 con 0x7f70b01068c0 2023-08-11T15:52:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b79ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b01a9760 con 0x7f70b01008b0 2023-08-11T15:52:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:58154/0 (socket says 172.21.15.32:58154) 2023-08-11T15:52:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b579b700 1 -- 172.21.15.32:0/2420616389 learned_addr learned my addr 172.21.15.32:0/2420616389 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70b01068c0 0x7f70b00736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.182+0000 7f70b4f9a700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b0101270 0x7f70b0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.183+0000 7f70b579b700 1 -- 172.21.15.32:0/2420616389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70b01068c0 msgr2=0x7f70b00736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.183+0000 7f70b579b700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70b01068c0 0x7f70b00736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.183+0000 7f70b579b700 1 -- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b0101270 msgr2=0x7f70b0073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.183+0000 7f70b579b700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b0101270 0x7f70b0073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.183+0000 7f70b579b700 1 -- 172.21.15.32:0/2420616389 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8009c40 con 0x7f70b01008b0 2023-08-11T15:52:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.183+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70b01068c0 0x7f70b00736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.183+0000 7f70b579b700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0072c20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e9a0 tx=0x7f70ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.184+0000 7f70a67fc700 1 -- 172.21.15.32:0/2420616389 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac009790 con 0x7f70b01008b0 2023-08-11T15:52:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.184+0000 7f70a67fc700 1 -- 172.21.15.32:0/2420616389 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac00ce80 con 0x7f70b01008b0 2023-08-11T15:52:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.184+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a00095f0 con 0x7f70b01008b0 2023-08-11T15:52:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.184+0000 7f70a67fc700 1 -- 172.21.15.32:0/2420616389 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac018680 con 0x7f70b01008b0 2023-08-11T15:52:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.184+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b01a9e10 con 0x7f70b01008b0 2023-08-11T15:52:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.188+0000 7f70a67fc700 1 -- 172.21.15.32:0/2420616389 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f70ac010430 con 0x7f70b01008b0 2023-08-11T15:52:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.189+0000 7f70cb31bb80 1 Processor -- start 2023-08-11T15:52:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70cb31bb80 1 -- start start 2023-08-11T15:52:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf494ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494ff0 0x562ebf4a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf3ed5d0 con 0x562ebf494ff0 2023-08-11T15:52:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf3eaef0 con 0x562ebf494630 2023-08-11T15:52:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf3e67e0 con 0x562ebf499e80 2023-08-11T15:52:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.190+0000 7f70a67fc700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f709c0a0140 0x7f709c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf494ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf494ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55150/0 (socket says 172.21.15.32:55150) 2023-08-11T15:52:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 -- 172.21.15.32:0/1177230591 learned_addr learned my addr 172.21.15.32:0/1177230591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70a67fc700 1 -- 172.21.15.32:0/2420616389 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f70ac014070 con 0x7f70b01008b0 2023-08-11T15:52:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b4f9a700 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494ff0 0x562ebf4a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f709c0a0140 0x7f709c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 -- 172.21.15.32:0/1177230591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 msgr2=0x562ebf49a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 -- 172.21.15.32:0/1177230591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494ff0 msgr2=0x562ebf4a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494ff0 0x562ebf4a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 -- 172.21.15.32:0/1177230591 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ebf2665d0 con 0x562ebf494630 2023-08-11T15:52:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b4f9a700 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494ff0 0x562ebf4a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.191+0000 7f70b579b700 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf494ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70ac095320 tx=0x7f70ac010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5bb64a70cafe896 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.192+0000 7f709b7fe700 1 -- 172.21.15.32:0/1177230591 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac097480 con 0x562ebf494630 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.192+0000 7f709b7fe700 1 -- 172.21.15.32:0/1177230591 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac099520 con 0x562ebf494630 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.192+0000 7f709b7fe700 1 -- 172.21.15.32:0/1177230591 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0b8070 con 0x562ebf494630 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1177230591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 msgr2=0x562ebf494ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf494ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70ac095320 tx=0x7f70ac010b10 comp rx=0 tx=0).stop 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f709c0a0140 0x7f709c0a2600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009fd0 tx=0x7f70a800b040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1177230591 shutdown_connections 2023-08-11T15:52:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494ff0 0x562ebf4a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1177230591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf494ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1177230591 >> 172.21.15.32:0/1177230591 conn(0x562ebf3c18f0 msgr2=0x562ebf48ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1177230591 shutdown_connections 2023-08-11T15:52:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1177230591 wait complete. 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.193+0000 7f70cb31bb80 1 -- start start 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.194+0000 7f70cb31bb80 1 Processor -- start 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.194+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf4a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.194+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.194+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.194+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf384a60 con 0x562ebf494630 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.194+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf384760 con 0x562ebf494ff0 2023-08-11T15:52:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.194+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf2688c0 con 0x562ebf499e80 2023-08-11T15:52:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.195+0000 7f70cb31bb80 10 client.0 _ll_register_callbacks cb 0x562ebf387a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.195+0000 7f70b5f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.195+0000 7f70b5f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:58174/0 (socket says 172.21.15.32:58174) 2023-08-11T15:52:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.195+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1023861049 learned_addr learned my addr 172.21.15.32:0/1023861049 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.195+0000 7f70b579b700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf4a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.195+0000 7f70b4f9a700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 msgr2=0x562ebf4a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1023861049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 msgr2=0x562ebf4a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf4a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1023861049 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b01a9e10 con 0x562ebf499e80 2023-08-11T15:52:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f70b579b700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf4a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49dcb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70a8003f80 tx=0x7f70a8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51a124ed0ead0db server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f7098ff9700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:52:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.196+0000 7f7098ff9700 1 -- 172.21.15.32:0/1023861049 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a800ee90 con 0x562ebf499e80 2023-08-11T15:52:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.197+0000 7f7098ff9700 1 -- 172.21.15.32:0/1023861049 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f70a8004260 con 0x562ebf499e80 2023-08-11T15:52:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.197+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ebf4877e0 con 0x562ebf499e80 2023-08-11T15:52:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.197+0000 7f7098ff9700 1 -- 172.21.15.32:0/1023861049 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70a8004420 con 0x562ebf499e80 2023-08-11T15:52:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.197+0000 7f70cb31bb80 10 client.25849 fetch_fsmap learned FSMap version 27 2023-08-11T15:52:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.197+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562ebf212d80 con 0x562ebf499e80 2023-08-11T15:52:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.197+0000 7f7098ff9700 1 -- 172.21.15.32:0/1023861049 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ebf4877e0 con 0x562ebf499e80 2023-08-11T15:52:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.197+0000 7f7098ff9700 1 -- 172.21.15.32:0/1023861049 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f70a8018c70 con 0x562ebf499e80 2023-08-11T15:52:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70cb31bb80 10 client.25849 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70cb31bb80 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70cb31bb80 20 client.25849 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf4adf00 0x562ebf4b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70cb31bb80 4 client.25849 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ebf463480 con 0x562ebf4adf00 2023-08-11T15:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70b579b700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf4adf00 0x562ebf4b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f70b579b700 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf4adf00 0x562ebf4b0320 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628fa6fafffa62e6 server_cookie=de1690cdbea2b9d8 in_seq=0 out_seq=0 2023-08-11T15:52:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.198+0000 7f7098ff9700 10 client.25849 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f7098ff9700 1 -- 172.21.15.32:0/1023861049 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562ebf463480 con 0x562ebf4adf00 2023-08-11T15:52:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f7098ff9700 10 client.25849 handle_command_reply: tid=0 2023-08-11T15:52:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 client.25849 shutdown 2023-08-11T15:52:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 msgr2=0x562ebf49dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49dcb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70a8003f80 tx=0x7f70a8004b40 comp rx=0 tx=0).stop 2023-08-11T15:52:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 shutdown_connections 2023-08-11T15:52:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf4adf00 0x562ebf4b0320 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf4a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf49dcb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1023861049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 >> 172.21.15.32:0/1023861049 conn(0x562ebf3c18f0 msgr2=0x562ebf48e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.201+0000 7f7098ff9700 0 client.25849 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.202+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 shutdown_connections 2023-08-11T15:52:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.202+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1023861049 wait complete. 2023-08-11T15:52:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.202+0000 7f70cb31bb80 20 client.25849 trim_cache size 0 max 16384 2023-08-11T15:52:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.219+0000 7f70cb31bb80 1 Processor -- start 2023-08-11T15:52:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.219+0000 7f70cb31bb80 1 -- start start 2023-08-11T15:52:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf49dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf48ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf4901a0 con 0x562ebf494630 2023-08-11T15:52:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf487570 con 0x562ebf494ff0 2023-08-11T15:52:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf49c230 con 0x562ebf499e80 2023-08-11T15:52:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70b5f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf48ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70b5f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf48ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:58184/0 (socket says 172.21.15.32:58184) 2023-08-11T15:52:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1663764002 learned_addr learned my addr 172.21.15.32:0/1663764002 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70b579b700 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf49dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.220+0000 7f70b4f9a700 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1663764002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 msgr2=0x562ebf4a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1663764002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 msgr2=0x562ebf49dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf49dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f70b5f9c700 1 -- 172.21.15.32:0/1663764002 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ebf1fbc00 con 0x562ebf499e80 2023-08-11T15:52:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f70b579b700 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf49dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf48ed50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f70a80077e0 tx=0x7f70a8007810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b16ae588fc7652c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f709bfff700 1 -- 172.21.15.32:0/1663764002 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8022a40 con 0x562ebf499e80 2023-08-11T15:52:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f709bfff700 1 -- 172.21.15.32:0/1663764002 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a800f040 con 0x562ebf499e80 2023-08-11T15:52:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.221+0000 7f709bfff700 1 -- 172.21.15.32:0/1663764002 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a80183f0 con 0x562ebf499e80 2023-08-11T15:52:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1663764002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 msgr2=0x562ebf48ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf48ed50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f70a80077e0 tx=0x7f70a8007810 comp rx=0 tx=0).stop 2023-08-11T15:52:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1663764002 shutdown_connections 2023-08-11T15:52:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf494630 0x562ebf49dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf499e80 0x562ebf48ed50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/1663764002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494ff0 0x562ebf4a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1663764002 >> 172.21.15.32:0/1663764002 conn(0x562ebf3c18f0 msgr2=0x562ebf49d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1663764002 shutdown_connections 2023-08-11T15:52:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 -- 172.21.15.32:0/1663764002 wait complete. 2023-08-11T15:52:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.222+0000 7f70cb31bb80 1 -- start start 2023-08-11T15:52:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 1 Processor -- start 2023-08-11T15:52:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf4faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf494ff0 0x562ebf4fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf499e80 0x562ebf4f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf4f6220 con 0x562ebf499e80 2023-08-11T15:52:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf4f6390 con 0x562ebf494630 2023-08-11T15:52:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ebf4f6550 con 0x562ebf494ff0 2023-08-11T15:52:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70cb31bb80 10 client.0 _ll_register_callbacks cb 0x7f70a800ede0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70b579b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf4faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70b4f9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf494ff0 0x562ebf4fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70b579b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf4faf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55188/0 (socket says 172.21.15.32:55188) 2023-08-11T15:52:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.223+0000 7f70b579b700 1 -- 172.21.15.32:0/2216423425 learned_addr learned my addr 172.21.15.32:0/2216423425 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70b5f9c700 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf499e80 0x562ebf4f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70b579b700 1 -- 172.21.15.32:0/2216423425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf494ff0 msgr2=0x562ebf4fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70b579b700 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf494ff0 0x562ebf4fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70b579b700 1 -- 172.21.15.32:0/2216423425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf499e80 msgr2=0x562ebf4f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70b579b700 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf499e80 0x562ebf4f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70b579b700 1 -- 172.21.15.32:0/2216423425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ebf212d80 con 0x562ebf494630 2023-08-11T15:52:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70b579b700 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf4faf60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f70ac097d80 tx=0x7f70ac095290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6412f9e7235f20ab server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f7098ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:52:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f7098ff9700 1 -- 172.21.15.32:0/2216423425 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0b80b0 con 0x562ebf494630 2023-08-11T15:52:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70ac099520 con 0x562ebf494630 2023-08-11T15:52:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.224+0000 7f7098ff9700 1 -- 172.21.15.32:0/2216423425 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f70ac0b8210 con 0x562ebf494630 2023-08-11T15:52:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f7098ff9700 1 -- 172.21.15.32:0/2216423425 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70ac099080 con 0x562ebf494630 2023-08-11T15:52:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 10 client.25853 fetch_fsmap learned FSMap version 27 2023-08-11T15:52:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562ebf2391f0 con 0x562ebf494630 2023-08-11T15:52:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f7098ff9700 1 -- 172.21.15.32:0/2216423425 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac099520 con 0x562ebf494630 2023-08-11T15:52:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f7098ff9700 1 -- 172.21.15.32:0/2216423425 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f70ac098310 con 0x562ebf494630 2023-08-11T15:52:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 10 client.25853 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:52:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 10 client.25853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 20 client.25853 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf502b10 0x562ebf504f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 4 client.25853 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:52:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.225+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ebf3df7c0 con 0x562ebf502b10 2023-08-11T15:52:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.226+0000 7f70b4f9a700 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf502b10 0x562ebf504f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.226+0000 7f70b4f9a700 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf502b10 0x562ebf504f00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2746ca68944560d server_cookie=791323a0cac573fa in_seq=0 out_seq=0 2023-08-11T15:52:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.226+0000 7f7098ff9700 10 client.25853 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.527+0000 7f7098ff9700 1 -- 172.21.15.32:0/2216423425 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ebf3df7c0 con 0x562ebf502b10 2023-08-11T15:52:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.527+0000 7f7098ff9700 10 client.25853 handle_command_reply: tid=0 2023-08-11T15:52:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.527+0000 7f70cb31bb80 1 client.25853 shutdown 2023-08-11T15:52:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 msgr2=0x562ebf4faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf4faf60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f70ac097d80 tx=0x7f70ac095290 comp rx=0 tx=0).stop 2023-08-11T15:52:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 shutdown_connections 2023-08-11T15:52:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562ebf502b10 0x562ebf504f00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562ebf499e80 0x562ebf4f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ebf494630 0x562ebf4faf60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 --2- 172.21.15.32:0/2216423425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ebf494ff0 0x562ebf4fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f7098ff9700 0 client.25853 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 >> 172.21.15.32:0/2216423425 conn(0x562ebf3c18f0 msgr2=0x562ebf498d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 shutdown_connections 2023-08-11T15:52:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.528+0000 7f70cb31bb80 1 -- 172.21.15.32:0/2216423425 wait complete. 2023-08-11T15:52:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.530+0000 7f70cb31bb80 20 client.25853 trim_cache size 0 max 16384 2023-08-11T15:52:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:52:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:52:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:52:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:52:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.531+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f709c0a0140 msgr2=0x7f709c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.531+0000 7f70b79ff700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f709c0a0140 0x7f709c0a2600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009fd0 tx=0x7f70a800b040 comp rx=0 tx=0).stop 2023-08-11T15:52:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 msgr2=0x7f70b0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0072c20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e9a0 tx=0x7f70ac00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 shutdown_connections 2023-08-11T15:52:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f709c0a0140 0x7f709c0a2600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70b01068c0 0x7f70b00736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70b01008b0 0x7f70b0072c20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 --2- 172.21.15.32:0/2420616389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b0101270 0x7f70b0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.532+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 >> 172.21.15.32:0/2420616389 conn(0x7f70b00fc380 msgr2=0x7f70b00fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.533+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 shutdown_connections 2023-08-11T15:52:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:20.533+0000 7f70b79ff700 1 -- 172.21.15.32:0/2420616389 wait complete. 2023-08-11T15:52:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:20 smithi192 ceph-mon[130806]: pgmap v175: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 348 op/s 2023-08-11T15:52:20.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:20 smithi134 ceph-mon[130447]: pgmap v175: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 348 op/s 2023-08-11T15:52:20.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 637228 11.89 MB/sec execute 156 sec latency 166.095 ms 2023-08-11T15:52:20.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:20 smithi032 ceph-mon[97103]: pgmap v175: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 348 op/s 2023-08-11T15:52:21.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 639642 11.90 MB/sec execute 157 sec latency 37.592 ms 2023-08-11T15:52:22.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:22 smithi192 ceph-mon[130806]: pgmap v176: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2023-08-11T15:52:22.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:22 smithi134 ceph-mon[130447]: pgmap v176: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2023-08-11T15:52:22.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 641747 11.88 MB/sec execute 158 sec latency 14.831 ms 2023-08-11T15:52:22.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:22 smithi032 ceph-mon[97103]: pgmap v176: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2023-08-11T15:52:23.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 643609 11.88 MB/sec execute 159 sec latency 38.927 ms 2023-08-11T15:52:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:24 smithi192 ceph-mon[130806]: pgmap v177: 129 pgs: 129 active+clean; 58 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2023-08-11T15:52:24.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:24 smithi134 ceph-mon[130447]: pgmap v177: 129 pgs: 129 active+clean; 58 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2023-08-11T15:52:24.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 646418 11.91 MB/sec execute 160 sec latency 23.127 ms 2023-08-11T15:52:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:24 smithi032 ceph-mon[97103]: pgmap v177: 129 pgs: 129 active+clean; 58 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2023-08-11T15:52:25.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 648045 11.88 MB/sec execute 161 sec latency 30.014 ms 2023-08-11T15:52:26.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:26 smithi192 ceph-mon[130806]: pgmap v178: 129 pgs: 129 active+clean; 58 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 293 op/s 2023-08-11T15:52:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:26 smithi134 ceph-mon[130447]: pgmap v178: 129 pgs: 129 active+clean; 58 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 293 op/s 2023-08-11T15:52:26.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 650557 11.89 MB/sec execute 162 sec latency 14.927 ms 2023-08-11T15:52:26.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:26 smithi032 ceph-mon[97103]: pgmap v178: 129 pgs: 129 active+clean; 58 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 293 op/s 2023-08-11T15:52:27.760 INFO:tasks.workunit.client.0.smithi032.stdout: 1 653195 11.91 MB/sec execute 163 sec latency 14.233 ms 2023-08-11T15:52:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:27 smithi032 ceph-mon[97103]: pgmap v179: 129 pgs: 129 active+clean; 64 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 349 op/s 2023-08-11T15:52:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:27 smithi192 ceph-mon[130806]: pgmap v179: 129 pgs: 129 active+clean; 64 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 349 op/s 2023-08-11T15:52:28.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:27 smithi134 ceph-mon[130447]: pgmap v179: 129 pgs: 129 active+clean; 64 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 349 op/s 2023-08-11T15:52:28.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 655049 11.89 MB/sec execute 164 sec latency 28.898 ms 2023-08-11T15:52:29.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 657322 11.91 MB/sec execute 165 sec latency 163.688 ms 2023-08-11T15:52:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:30 smithi192 ceph-mon[130806]: pgmap v180: 129 pgs: 129 active+clean; 66 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2023-08-11T15:52:30.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:30 smithi134 ceph-mon[130447]: pgmap v180: 129 pgs: 129 active+clean; 66 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2023-08-11T15:52:30.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 659330 11.88 MB/sec execute 166 sec latency 60.010 ms 2023-08-11T15:52:30.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:30 smithi032 ceph-mon[97103]: pgmap v180: 129 pgs: 129 active+clean; 66 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2023-08-11T15:52:31.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 661356 11.88 MB/sec execute 167 sec latency 11.678 ms 2023-08-11T15:52:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:32 smithi192 ceph-mon[130806]: pgmap v181: 129 pgs: 129 active+clean; 66 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 295 op/s 2023-08-11T15:52:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:32 smithi134 ceph-mon[130447]: pgmap v181: 129 pgs: 129 active+clean; 66 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 295 op/s 2023-08-11T15:52:32.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 664064 11.91 MB/sec execute 168 sec latency 14.611 ms 2023-08-11T15:52:32.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:32 smithi032 ceph-mon[97103]: pgmap v181: 129 pgs: 129 active+clean; 66 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 295 op/s 2023-08-11T15:52:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 -- 172.21.15.32:0/1909129573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a40101260 msgr2=0x7f2a4010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1909129573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a40101260 0x7f2a4010d770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b0a0 tx=0x7f2a3800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 -- 172.21.15.32:0/1909129573 shutdown_connections 2023-08-11T15:52:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1909129573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a40101260 0x7f2a4010d770 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1909129573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a401068b0 0x7f2a40106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1909129573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a401008a0 0x7f2a40100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 -- 172.21.15.32:0/1909129573 >> 172.21.15.32:0/1909129573 conn(0x7f2a400fc370 msgr2=0x7f2a400fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 -- 172.21.15.32:0/1909129573 shutdown_connections 2023-08-11T15:52:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.698+0000 7f2a455ff700 1 -- 172.21.15.32:0/1909129573 wait complete. 2023-08-11T15:52:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.699+0000 7f2a455ff700 1 Processor -- start 2023-08-11T15:52:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.699+0000 7f2a455ff700 1 -- start start 2023-08-11T15:52:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.699+0000 7f2a455ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a401008a0 0x7f2a40072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a455ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a40101260 0x7f2a40073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a455ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a401068b0 0x7f2a401a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a455ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401a4f60 con 0x7f2a401008a0 2023-08-11T15:52:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a455ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401a50a0 con 0x7f2a40101260 2023-08-11T15:52:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a455ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401a5210 con 0x7f2a401068b0 2023-08-11T15:52:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a40101260 0x7f2a40073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a401008a0 0x7f2a40072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a40101260 0x7f2a40073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:47622/0 (socket says 172.21.15.32:47622) 2023-08-11T15:52:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/1781324713 learned_addr learned my addr 172.21.15.32:0/1781324713 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3f7fe700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a401068b0 0x7f2a401a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/1781324713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a401068b0 msgr2=0x7f2a401a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a401068b0 0x7f2a401a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a401008a0 msgr2=0x7f2a40072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a401008a0 0x7f2a40072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/1781324713 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30009520 con 0x7f2a40101260 2023-08-11T15:52:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.700+0000 7f2a3effd700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a401008a0 0x7f2a40072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.701+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a40101260 0x7f2a40073340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9a0 tx=0x7f2a3400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.701+0000 7f2a27fff700 1 -- 172.21.15.32:0/1781324713 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34009790 con 0x7f2a40101260 2023-08-11T15:52:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.701+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a38009d00 con 0x7f2a40101260 2023-08-11T15:52:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.701+0000 7f2a27fff700 1 -- 172.21.15.32:0/1781324713 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3400cbd0 con 0x7f2a40101260 2023-08-11T15:52:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.701+0000 7f2a27fff700 1 -- 172.21.15.32:0/1781324713 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34018680 con 0x7f2a40101260 2023-08-11T15:52:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.701+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a401a58c0 con 0x7f2a40101260 2023-08-11T15:52:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.704+0000 7f2a27fff700 1 -- 172.21.15.32:0/1781324713 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f2a3400cd40 con 0x7f2a40101260 2023-08-11T15:52:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.705+0000 7f2a27fff700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2a280a0140 0x7f2a280a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 Processor -- start 2023-08-11T15:52:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a27fff700 1 -- 172.21.15.32:0/1781324713 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2a34014070 con 0x7f2a40101260 2023-08-11T15:52:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a3effd700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2a280a0140 0x7f2a280a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 -- start start 2023-08-11T15:52:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbf5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc01c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cb4e5d0 con 0x55f06cbf5ff0 2023-08-11T15:52:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cb4bef0 con 0x55f06cbfae80 2023-08-11T15:52:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.706+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cb477e0 con 0x55f06cbf5630 2023-08-11T15:52:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbf5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3effd700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2a280a0140 0x7f2a280a2600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a30011630 tx=0x7f2a30009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbf5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:41330/0 (socket says 172.21.15.32:41330) 2023-08-11T15:52:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/2482239386 learned_addr learned my addr 172.21.15.32:0/2482239386 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3f7fe700 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc01c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/2482239386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 msgr2=0x55f06cbfb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/2482239386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 msgr2=0x55f06cc01c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc01c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/2482239386 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f06c9c75d0 con 0x55f06cbf5630 2023-08-11T15:52:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.707+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbf5ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a340a0870 tx=0x7f2a340988f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ef2a4299c493a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a257fa700 1 -- 172.21.15.32:0/2482239386 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3409cdc0 con 0x55f06cbf5630 2023-08-11T15:52:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a257fa700 1 -- 172.21.15.32:0/2482239386 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a340a0da0 con 0x55f06cbf5630 2023-08-11T15:52:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a257fa700 1 -- 172.21.15.32:0/2482239386 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340a0910 con 0x55f06cbf5630 2023-08-11T15:52:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/2482239386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 msgr2=0x55f06cbf5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbf5ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a340a0870 tx=0x7f2a340988f0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/2482239386 shutdown_connections 2023-08-11T15:52:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc01c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/2482239386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbf5ab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.708+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/2482239386 >> 172.21.15.32:0/2482239386 conn(0x55f06cb228f0 msgr2=0x55f06cbefe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.709+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/2482239386 shutdown_connections 2023-08-11T15:52:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.709+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/2482239386 wait complete. 2023-08-11T15:52:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.709+0000 7f2a58eb3b80 1 -- start start 2023-08-11T15:52:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.709+0000 7f2a58eb3b80 1 Processor -- start 2023-08-11T15:52:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cc049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc04ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cae5a60 con 0x55f06cbf5ff0 2023-08-11T15:52:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cae5760 con 0x55f06cbfae80 2023-08-11T15:52:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06c9c98c0 con 0x55f06cbf5630 2023-08-11T15:52:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a58eb3b80 10 client.0 _ll_register_callbacks cb 0x55f06cae8a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cc049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cc049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:41344/0 (socket says 172.21.15.32:41344) 2023-08-11T15:52:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3effd700 1 -- 172.21.15.32:0/1481015580 learned_addr learned my addr 172.21.15.32:0/1481015580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3f7fe700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc04ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3effd700 1 -- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 msgr2=0x55f06cbfeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3effd700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3effd700 1 -- 172.21.15.32:0/1481015580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 msgr2=0x55f06cc04ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.710+0000 7f2a3effd700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc04ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a3effd700 1 -- 172.21.15.32:0/1481015580 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a401a58c0 con 0x55f06cbf5630 2023-08-11T15:52:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc04ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a3effd700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cc049b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a3001db80 tx=0x7f2a300047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de7e18fad74f87d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:52:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a22ff5700 1 -- 172.21.15.32:0/1481015580 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30023070 con 0x55f06cbf5630 2023-08-11T15:52:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f06cbe87e0 con 0x55f06cbf5630 2023-08-11T15:52:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a22ff5700 1 -- 172.21.15.32:0/1481015580 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2a3001ea40 con 0x55f06cbf5630 2023-08-11T15:52:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.711+0000 7f2a22ff5700 1 -- 172.21.15.32:0/1481015580 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a3001ec40 con 0x55f06cbf5630 2023-08-11T15:52:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 10 client.25867 fetch_fsmap learned FSMap version 27 2023-08-11T15:52:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f06c973d80 con 0x55f06cbf5630 2023-08-11T15:52:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a22ff5700 1 -- 172.21.15.32:0/1481015580 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f06cbe87e0 con 0x55f06cbf5630 2023-08-11T15:52:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a22ff5700 1 -- 172.21.15.32:0/1481015580 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2a3001b430 con 0x55f06cbf5630 2023-08-11T15:52:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 10 client.25867 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:52:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 20 client.25867 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc0ef30 0x55f06cc11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 4 client.25867 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:52:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f06cbc4480 con 0x55f06cc0ef30 2023-08-11T15:52:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.712+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc0ef30 0x55f06cc11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.714+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc0ef30 0x55f06cc11350 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555f6dd382ca57ce server_cookie=a287a729e5c34266 in_seq=0 out_seq=0 2023-08-11T15:52:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.714+0000 7f2a22ff5700 10 client.25867 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a22ff5700 1 -- 172.21.15.32:0/1481015580 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f06cbc4480 con 0x55f06cc0ef30 2023-08-11T15:52:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a22ff5700 10 client.25867 handle_command_reply: tid=0 2023-08-11T15:52:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 client.25867 shutdown 2023-08-11T15:52:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 msgr2=0x55f06cc049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cc049b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a3001db80 tx=0x7f2a300047c0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 shutdown_connections 2023-08-11T15:52:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc0ef30 0x55f06cc11350 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbfeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cc049b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/1481015580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc04ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a22ff5700 0 client.25867 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 >> 172.21.15.32:0/1481015580 conn(0x55f06cb228f0 msgr2=0x55f06cbef5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 shutdown_connections 2023-08-11T15:52:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/1481015580 wait complete. 2023-08-11T15:52:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.715+0000 7f2a58eb3b80 20 client.25867 trim_cache size 0 max 16384 2023-08-11T15:52:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.730+0000 7f2a58eb3b80 1 Processor -- start 2023-08-11T15:52:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.730+0000 7f2a58eb3b80 1 -- start start 2023-08-11T15:52:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbfb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbefd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cbf63d0 con 0x55f06cbf5ff0 2023-08-11T15:52:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cc00280 con 0x55f06cbfae80 2023-08-11T15:52:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cbf1d50 con 0x55f06cbf5630 2023-08-11T15:52:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbfb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbefd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbfb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:41352/0 (socket says 172.21.15.32:41352) 2023-08-11T15:52:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 -- 172.21.15.32:0/4178382099 learned_addr learned my addr 172.21.15.32:0/4178382099 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3f7fe700 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbefd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:47642/0 (socket says 172.21.15.32:47642) 2023-08-11T15:52:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 -- 172.21.15.32:0/4178382099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 msgr2=0x55f06cbefd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbefd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 -- 172.21.15.32:0/4178382099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 msgr2=0x55f06cc044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.731+0000 7f2a3effd700 1 -- 172.21.15.32:0/4178382099 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f06c95cc00 con 0x55f06cbf5630 2023-08-11T15:52:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.732+0000 7f2a3effd700 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbfb260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2a30005d90 tx=0x7f2a30009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b606087fb57886 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.732+0000 7f2a25ffb700 1 -- 172.21.15.32:0/4178382099 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30023070 con 0x55f06cbf5630 2023-08-11T15:52:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.732+0000 7f2a25ffb700 1 -- 172.21.15.32:0/4178382099 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a30011d80 con 0x55f06cbf5630 2023-08-11T15:52:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.732+0000 7f2a25ffb700 1 -- 172.21.15.32:0/4178382099 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300366c0 con 0x55f06cbf5630 2023-08-11T15:52:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/4178382099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 msgr2=0x55f06cbfb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbfb260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2a30005d90 tx=0x7f2a30009370 comp rx=0 tx=0).stop 2023-08-11T15:52:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/4178382099 shutdown_connections 2023-08-11T15:52:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbfae80 0x55f06cbefd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbf5630 0x55f06cbfb260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/4178382099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/4178382099 >> 172.21.15.32:0/4178382099 conn(0x55f06cb228f0 msgr2=0x55f06cbf8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/4178382099 shutdown_connections 2023-08-11T15:52:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/4178382099 wait complete. 2023-08-11T15:52:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.733+0000 7f2a58eb3b80 1 -- start start 2023-08-11T15:52:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.734+0000 7f2a58eb3b80 1 Processor -- start 2023-08-11T15:52:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.734+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbf5630 0x55f06cc5f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.734+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc5fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.734+0000 7f2a58eb3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbfae80 0x55f06cc6a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.734+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cc60240 con 0x55f06cbf5ff0 2023-08-11T15:52:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.734+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cc6a970 con 0x55f06cbf5630 2023-08-11T15:52:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.734+0000 7f2a58eb3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06cc6ab30 con 0x55f06cbfae80 2023-08-11T15:52:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a58eb3b80 10 client.0 _ll_register_callbacks cb 0x7f2a3001d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbf5630 0x55f06cc5f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbf5630 0x55f06cc5f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:47656/0 (socket says 172.21.15.32:47656) 2023-08-11T15:52:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3effd700 1 -- 172.21.15.32:0/314462345 learned_addr learned my addr 172.21.15.32:0/314462345 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc5fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3f7fe700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbfae80 0x55f06cc6a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/314462345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbfae80 msgr2=0x55f06cc6a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbfae80 0x55f06cc6a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbf5630 msgr2=0x55f06cc5f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbf5630 0x55f06cc5f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3e7fc700 1 -- 172.21.15.32:0/314462345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f06c973d80 con 0x55f06cbf5ff0 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.735+0000 7f2a3f7fe700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbfae80 0x55f06cc6a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a3effd700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbf5630 0x55f06cc5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a3e7fc700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc5fab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a34099840 tx=0x7f2a340c9570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487694fb2084d27b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a22ff5700 1 -- 172.21.15.32:0/314462345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340993f0 con 0x55f06cbf5ff0 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a22ff5700 1 -- 172.21.15.32:0/314462345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f2a34098480 con 0x55f06cbf5ff0 2023-08-11T15:52:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a340a0da0 con 0x55f06cbf5ff0 2023-08-11T15:52:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a22ff5700 1 -- 172.21.15.32:0/314462345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a34098640 con 0x55f06cbf5ff0 2023-08-11T15:52:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a58eb3b80 10 client.15855 fetch_fsmap learned FSMap version 27 2023-08-11T15:52:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.736+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f06c99a1f0 con 0x55f06cbf5ff0 2023-08-11T15:52:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a22ff5700 1 -- 172.21.15.32:0/314462345 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a340a0da0 con 0x55f06cbf5ff0 2023-08-11T15:52:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a22ff5700 1 -- 172.21.15.32:0/314462345 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2a34094d80 con 0x55f06cbf5ff0 2023-08-11T15:52:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a58eb3b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T15:52:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a58eb3b80 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a58eb3b80 20 client.15855 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc56e90 0x55f06cc592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a58eb3b80 4 client.15855 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:52:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f06cb407c0 con 0x55f06cc56e90 2023-08-11T15:52:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.737+0000 7f2a3effd700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc56e90 0x55f06cc592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.738+0000 7f2a3effd700 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc56e90 0x55f06cc592b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cd2cf8a152d261 server_cookie=f5a4be57cf09db8b in_seq=0 out_seq=0 2023-08-11T15:52:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:33.738+0000 7f2a22ff5700 10 client.15855 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:33.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 665566 11.88 MB/sec execute 169 sec latency 53.011 ms 2023-08-11T15:52:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.008+0000 7f2a22ff5700 1 -- 172.21.15.32:0/314462345 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f06cb407c0 con 0x55f06cc56e90 2023-08-11T15:52:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.008+0000 7f2a22ff5700 10 client.15855 handle_command_reply: tid=0 2023-08-11T15:52:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.008+0000 7f2a58eb3b80 1 client.15855 shutdown 2023-08-11T15:52:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.010+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 msgr2=0x55f06cc5fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.010+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc5fab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a34099840 tx=0x7f2a340c9570 comp rx=0 tx=0).stop 2023-08-11T15:52:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 shutdown_connections 2023-08-11T15:52:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55f06cc56e90 0x55f06cc592b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f06cbfae80 0x55f06cc6a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f06cbf5630 0x55f06cc5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a58eb3b80 1 --2- 172.21.15.32:0/314462345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f06cbf5ff0 0x55f06cc5fab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 >> 172.21.15.32:0/314462345 conn(0x55f06cb228f0 msgr2=0x55f06cbf9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a22ff5700 0 client.15855 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.011+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 shutdown_connections 2023-08-11T15:52:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.012+0000 7f2a58eb3b80 1 -- 172.21.15.32:0/314462345 wait complete. 2023-08-11T15:52:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.012+0000 7f2a58eb3b80 20 client.15855 trim_cache size 0 max 16384 2023-08-11T15:52:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:52:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:52:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:52:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:52:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2a280a0140 msgr2=0x7f2a280a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2a280a0140 0x7f2a280a2600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a30011630 tx=0x7f2a30009260 comp rx=0 tx=0).stop 2023-08-11T15:52:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a40101260 msgr2=0x7f2a40073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a40101260 0x7f2a40073340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9a0 tx=0x7f2a3400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 shutdown_connections 2023-08-11T15:52:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f2a280a0140 0x7f2a280a2600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a401008a0 0x7f2a40072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a401068b0 0x7f2a401a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 --2- 172.21.15.32:0/1781324713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a40101260 0x7f2a40073340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 >> 172.21.15.32:0/1781324713 conn(0x7f2a400fc370 msgr2=0x7f2a400fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 shutdown_connections 2023-08-11T15:52:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:34.013+0000 7f2a455ff700 1 -- 172.21.15.32:0/1781324713 wait complete. 2023-08-11T15:52:34.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:34 smithi192 ceph-mon[130806]: pgmap v182: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 366 op/s 2023-08-11T15:52:34.628 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:34 smithi032 ceph-mon[97103]: pgmap v182: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 366 op/s 2023-08-11T15:52:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:34 smithi134 ceph-mon[130447]: pgmap v182: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 366 op/s 2023-08-11T15:52:34.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 666949 11.85 MB/sec execute 170 sec latency 206.794 ms 2023-08-11T15:52:35.053 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:52:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setuser ceph since I am not root 2023-08-11T15:52:35.054 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:52:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: ignoring --setgroup ceph since I am not root 2023-08-11T15:52:35.054 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:52:34 smithi032 ceph-mds[131163]: -- 172.21.15.32:0/452226115 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5572904191e0 con 0x5572903ee800 2023-08-11T15:52:35.054 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:52:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: starting mds.a at 2023-08-11T15:52:35.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:35 smithi192 ceph-mon[130806]: daemon mds.a restarted 2023-08-11T15:52:35.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:35 smithi134 ceph-mon[130447]: daemon mds.a restarted 2023-08-11T15:52:35.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 668777 11.84 MB/sec execute 171 sec latency 33.070 ms 2023-08-11T15:52:35.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:35 smithi032 ceph-mon[97103]: daemon mds.a restarted 2023-08-11T15:52:36.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:36 smithi192 ceph-mon[130806]: pgmap v183: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2023-08-11T15:52:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:36 smithi192 ceph-mon[130806]: osdmap e97: 12 total, 12 up, 12 in 2023-08-11T15:52:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:36 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.32:6840/3095427395,v1:172.21.15.32:6841/3095427395] up:boot 2023-08-11T15:52:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:36 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:52:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:36 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:52:36.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:36 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:52:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:36 smithi134 ceph-mon[130447]: pgmap v183: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2023-08-11T15:52:36.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:36 smithi134 ceph-mon[130447]: osdmap e97: 12 total, 12 up, 12 in 2023-08-11T15:52:36.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:36 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.32:6840/3095427395,v1:172.21.15.32:6841/3095427395] up:boot 2023-08-11T15:52:36.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:36 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:52:36.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:36 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:52:36.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:36 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:52:36.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 671451 11.86 MB/sec execute 172 sec latency 15.052 ms 2023-08-11T15:52:36.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:36 smithi032 ceph-mon[97103]: pgmap v183: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2023-08-11T15:52:36.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:36 smithi032 ceph-mon[97103]: osdmap e97: 12 total, 12 up, 12 in 2023-08-11T15:52:36.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:36 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.32:6840/3095427395,v1:172.21.15.32:6841/3095427395] up:boot 2023-08-11T15:52:36.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:36 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:52:36.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:36 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T15:52:36.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:36 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:52:37.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 673309 11.83 MB/sec execute 173 sec latency 54.694 ms 2023-08-11T15:52:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:37 smithi032 ceph-mon[97103]: pgmap v185: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-08-11T15:52:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:37 smithi192 ceph-mon[130806]: pgmap v185: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-08-11T15:52:38.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:37 smithi134 ceph-mon[130447]: pgmap v185: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-08-11T15:52:38.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 675521 11.84 MB/sec execute 174 sec latency 15.658 ms 2023-08-11T15:52:39.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 677973 11.85 MB/sec execute 175 sec latency 43.544 ms 2023-08-11T15:52:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:40 smithi192 ceph-mon[130806]: pgmap v186: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-08-11T15:52:40.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:40 smithi134 ceph-mon[130447]: pgmap v186: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-08-11T15:52:40.764 INFO:tasks.workunit.client.0.smithi032.stdout: 1 679531 11.83 MB/sec execute 176 sec latency 68.252 ms 2023-08-11T15:52:40.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:40 smithi032 ceph-mon[97103]: pgmap v186: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-08-11T15:52:41.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 681626 11.83 MB/sec execute 177 sec latency 266.078 ms 2023-08-11T15:52:42.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:42 smithi192 ceph-mon[130806]: pgmap v187: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-08-11T15:52:42.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:42 smithi134 ceph-mon[130447]: pgmap v187: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-08-11T15:52:42.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 683506 11.82 MB/sec execute 178 sec latency 13.641 ms 2023-08-11T15:52:42.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:42 smithi032 ceph-mon[97103]: pgmap v187: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2023-08-11T15:52:43.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 684620 11.78 MB/sec execute 179 sec latency 186.127 ms 2023-08-11T15:52:44.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:44 smithi192 ceph-mon[130806]: pgmap v188: 129 pgs: 129 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2023-08-11T15:52:44.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:44 smithi134 ceph-mon[130447]: pgmap v188: 129 pgs: 129 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2023-08-11T15:52:44.761 INFO:tasks.workunit.client.0.smithi032.stdout: 1 686137 11.77 MB/sec execute 180 sec latency 56.527 ms 2023-08-11T15:52:44.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:44 smithi032 ceph-mon[97103]: pgmap v188: 129 pgs: 129 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2023-08-11T15:52:45.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 688397 11.77 MB/sec execute 181 sec latency 41.294 ms 2023-08-11T15:52:46.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:46 smithi192 ceph-mon[130806]: pgmap v189: 129 pgs: 129 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2023-08-11T15:52:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:46 smithi134 ceph-mon[130447]: pgmap v189: 129 pgs: 129 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2023-08-11T15:52:46.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 690383 11.77 MB/sec execute 182 sec latency 34.326 ms 2023-08-11T15:52:46.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:46 smithi032 ceph-mon[97103]: pgmap v189: 129 pgs: 129 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2023-08-11T15:52:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 -- 172.21.15.32:0/553887156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 msgr2=0x7fcd28108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/553887156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 0x7fcd28108f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009990 tx=0x7fcd1c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 -- 172.21.15.32:0/553887156 shutdown_connections 2023-08-11T15:52:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/553887156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 0x7fcd28108f80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/553887156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd281034c0 0x7fcd2810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/553887156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd28102b00 0x7fcd28102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 -- 172.21.15.32:0/553887156 >> 172.21.15.32:0/553887156 conn(0x7fcd280fe5d0 msgr2=0x7fcd281009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 -- 172.21.15.32:0/553887156 shutdown_connections 2023-08-11T15:52:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.175+0000 7fcd2ebff700 1 -- 172.21.15.32:0/553887156 wait complete. 2023-08-11T15:52:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.176+0000 7fcd2ebff700 1 Processor -- start 2023-08-11T15:52:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.176+0000 7fcd2ebff700 1 -- start start 2023-08-11T15:52:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2ebff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28102b00 0x7fcd2819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2ebff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd281034c0 0x7fcd2819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2ebff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 0x7fcd281989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2ebff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd28199050 con 0x7fcd28108ba0 2023-08-11T15:52:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2ebff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd281991c0 con 0x7fcd28102b00 2023-08-11T15:52:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2ebff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd28199380 con 0x7fcd281034c0 2023-08-11T15:52:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28102b00 0x7fcd2819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28102b00 0x7fcd2819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:58290/0 (socket says 172.21.15.32:58290) 2023-08-11T15:52:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 -- 172.21.15.32:0/874962092 learned_addr learned my addr 172.21.15.32:0/874962092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 0x7fcd281989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd27fff700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd281034c0 0x7fcd2819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 -- 172.21.15.32:0/874962092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd281034c0 msgr2=0x7fcd2819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd281034c0 0x7fcd2819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 -- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 msgr2=0x7fcd281989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 0x7fcd281989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.177+0000 7fcd2c99b700 1 -- 172.21.15.32:0/874962092 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0095f0 con 0x7fcd28102b00 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.178+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 0x7fcd281989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.178+0000 7fcd2c99b700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28102b00 0x7fcd2819eb80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c005910 tx=0x7fcd1c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.178+0000 7fcd25ffb700 1 -- 172.21.15.32:0/874962092 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c01d070 con 0x7fcd28102b00 2023-08-11T15:52:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.178+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd28199600 con 0x7fcd28102b00 2023-08-11T15:52:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.178+0000 7fcd25ffb700 1 -- 172.21.15.32:0/874962092 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd1c022470 con 0x7fcd28102b00 2023-08-11T15:52:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.178+0000 7fcd25ffb700 1 -- 172.21.15.32:0/874962092 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c017650 con 0x7fcd28102b00 2023-08-11T15:52:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.178+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd281a5b70 con 0x7fcd28102b00 2023-08-11T15:52:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.181+0000 7fcd25ffb700 1 -- 172.21.15.32:0/874962092 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fcd1c0225e0 con 0x7fcd28102b00 2023-08-11T15:52:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.182+0000 7fcd42592b80 1 Processor -- start 2023-08-11T15:52:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd42592b80 1 -- start start 2023-08-11T15:52:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a5fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a6bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd42592b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde29b85d0 con 0x55dde2a5f630 2023-08-11T15:52:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd42592b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde29b5ef0 con 0x55dde2a5fff0 2023-08-11T15:52:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd42592b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde29b17e0 con 0x55dde2a64e80 2023-08-11T15:52:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd2c99b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a5fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a6bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd2d19c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a6bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:58298/0 (socket says 172.21.15.32:58298) 2023-08-11T15:52:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.183+0000 7fcd27fff700 1 -- 172.21.15.32:0/3758702438 learned_addr learned my addr 172.21.15.32:0/3758702438 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd2d19c700 1 -- 172.21.15.32:0/3758702438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 msgr2=0x55dde2a6bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/3758702438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a6bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd2d19c700 1 -- 172.21.15.32:0/3758702438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 msgr2=0x55dde2a5fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/3758702438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a5fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd2d19c700 1 -- 172.21.15.32:0/3758702438 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dde28315d0 con 0x55dde2a64e80 2023-08-11T15:52:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd25ffb700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcd180a00c0 0x7fcd180a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd25ffb700 1 -- 172.21.15.32:0/874962092 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd1c0ccd80 con 0x7fcd28102b00 2023-08-11T15:52:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/3758702438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a65260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009370 tx=0x7fcd20009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22fab8a568f675df server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.184+0000 7fcd2c99b700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcd180a00c0 0x7fcd180a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd12ffd700 1 -- 172.21.15.32:0/3758702438 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000e070 con 0x55dde2a64e80 2023-08-11T15:52:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd12ffd700 1 -- 172.21.15.32:0/3758702438 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2000f040 con 0x55dde2a64e80 2023-08-11T15:52:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd12ffd700 1 -- 172.21.15.32:0/3758702438 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20015ea0 con 0x55dde2a64e80 2023-08-11T15:52:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd42592b80 1 -- 172.21.15.32:0/3758702438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 msgr2=0x55dde2a65260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd2c99b700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcd180a00c0 0x7fcd180a2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55dde2a5ae90 tx=0x7fcd1c022f90 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3758702438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a65260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009370 tx=0x7fcd20009920 comp rx=0 tx=0).stop 2023-08-11T15:52:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd42592b80 1 -- 172.21.15.32:0/3758702438 shutdown_connections 2023-08-11T15:52:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3758702438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a6bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3758702438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a65260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.185+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3758702438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a5fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.186+0000 7fcd42592b80 1 -- 172.21.15.32:0/3758702438 >> 172.21.15.32:0/3758702438 conn(0x55dde298c8f0 msgr2=0x55dde2a59e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.186+0000 7fcd42592b80 1 -- 172.21.15.32:0/3758702438 shutdown_connections 2023-08-11T15:52:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.186+0000 7fcd42592b80 1 -- 172.21.15.32:0/3758702438 wait complete. 2023-08-11T15:52:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.186+0000 7fcd42592b80 1 -- start start 2023-08-11T15:52:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.186+0000 7fcd42592b80 1 Processor -- start 2023-08-11T15:52:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2a68d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd42592b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde294fa60 con 0x55dde2a64e80 2023-08-11T15:52:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd42592b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde294f760 con 0x55dde2a5f630 2023-08-11T15:52:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd42592b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde28338c0 con 0x55dde2a5fff0 2023-08-11T15:52:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd42592b80 10 client.0 _ll_register_callbacks cb 0x55dde2952a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd2d19c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd2d19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2a68d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd2d19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2a68d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56110/0 (socket says 172.21.15.32:56110) 2023-08-11T15:52:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.187+0000 7fcd2d19c700 1 -- 172.21.15.32:0/3626893589 learned_addr learned my addr 172.21.15.32:0/3626893589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd27fff700 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd2d19c700 1 -- 172.21.15.32:0/3626893589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 msgr2=0x55dde2a6ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a6ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd2d19c700 1 -- 172.21.15.32:0/3626893589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 msgr2=0x55dde2a6e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a6e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd2d19c700 1 -- 172.21.15.32:0/3626893589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd20009050 con 0x55dde2a64e80 2023-08-11T15:52:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2a68d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007ac0 tx=0x7fcd20000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef77b6e5fb365841 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd07fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:52:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.188+0000 7fcd07fff700 1 -- 172.21.15.32:0/3626893589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000e040 con 0x55dde2a64e80 2023-08-11T15:52:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dde2a527e0 con 0x55dde2a64e80 2023-08-11T15:52:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd07fff700 1 -- 172.21.15.32:0/3626893589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd2001e430 con 0x55dde2a64e80 2023-08-11T15:52:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd07fff700 1 -- 172.21.15.32:0/3626893589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2001e5f0 con 0x55dde2a64e80 2023-08-11T15:52:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd42592b80 10 client.15873 fetch_fsmap learned FSMap version 29 2023-08-11T15:52:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dde27ddd80 con 0x55dde2a64e80 2023-08-11T15:52:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd07fff700 1 -- 172.21.15.32:0/3626893589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dde2a527e0 con 0x55dde2a64e80 2023-08-11T15:52:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd07fff700 1 -- 172.21.15.32:0/3626893589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd200204b0 con 0x55dde2a64e80 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd42592b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.189+0000 7fcd42592b80 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.190+0000 7fcd42592b80 20 client.15873 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.190+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2a78f40 0x55dde2a7b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.190+0000 7fcd42592b80 4 client.15873 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.190+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dde2a2e480 con 0x55dde2a78f40 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.190+0000 7fcd27fff700 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2a78f40 0x55dde2a7b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.190+0000 7fcd27fff700 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2a78f40 0x55dde2a7b360 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87d9d8dfbf2f5a9 server_cookie=c6febb40df777ec0 in_seq=0 out_seq=0 2023-08-11T15:52:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.190+0000 7fcd07fff700 10 client.15873 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd07fff700 1 -- 172.21.15.32:0/3626893589 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55dde2a2e480 con 0x55dde2a78f40 2023-08-11T15:52:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd07fff700 10 client.15873 handle_command_reply: tid=0 2023-08-11T15:52:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 client.15873 shutdown 2023-08-11T15:52:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 msgr2=0x55dde2a68d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2a68d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007ac0 tx=0x7fcd20000f40 comp rx=0 tx=0).stop 2023-08-11T15:52:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 shutdown_connections 2023-08-11T15:52:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2a78f40 0x55dde2a7b360 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2a68d40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a6e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd07fff700 0 client.15873 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 --2- 172.21.15.32:0/3626893589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a6ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.191+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 >> 172.21.15.32:0/3626893589 conn(0x55dde298c8f0 msgr2=0x55dde2a595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.192+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 shutdown_connections 2023-08-11T15:52:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.192+0000 7fcd42592b80 1 -- 172.21.15.32:0/3626893589 wait complete. 2023-08-11T15:52:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.192+0000 7fcd42592b80 20 client.15873 trim_cache size 0 max 16384 2023-08-11T15:52:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 Processor -- start 2023-08-11T15:52:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 -- start start 2023-08-11T15:52:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a59d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde2a5bdb0 con 0x55dde2a5f630 2023-08-11T15:52:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde2a7b9c0 con 0x55dde2a5fff0 2023-08-11T15:52:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd42592b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde2a639d0 con 0x55dde2a64e80 2023-08-11T15:52:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd2d19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a59d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:58308/0 (socket says 172.21.15.32:58308) 2023-08-11T15:52:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.209+0000 7fcd27fff700 1 -- 172.21.15.32:0/1849035250 learned_addr learned my addr 172.21.15.32:0/1849035250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd27fff700 1 -- 172.21.15.32:0/1849035250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 msgr2=0x55dde2a682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd27fff700 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd27fff700 1 -- 172.21.15.32:0/1849035250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 msgr2=0x55dde2a7fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd27fff700 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a7fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd27fff700 1 -- 172.21.15.32:0/1849035250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dde27c6c00 con 0x55dde2a5fff0 2023-08-11T15:52:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a7fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:52:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd27fff700 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a59d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd140095c0 tx=0x7fcd14009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0afa3f253adf28f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd137fe700 1 -- 172.21.15.32:0/1849035250 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14016070 con 0x55dde2a5fff0 2023-08-11T15:52:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd137fe700 1 -- 172.21.15.32:0/1849035250 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd14017040 con 0x55dde2a5fff0 2023-08-11T15:52:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.210+0000 7fcd137fe700 1 -- 172.21.15.32:0/1849035250 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd140107b0 con 0x55dde2a5fff0 2023-08-11T15:52:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 -- 172.21.15.32:0/1849035250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 msgr2=0x55dde2a59d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a59d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd140095c0 tx=0x7fcd14009260 comp rx=0 tx=0).stop 2023-08-11T15:52:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 -- 172.21.15.32:0/1849035250 shutdown_connections 2023-08-11T15:52:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a5f630 0x55dde2a7fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a64e80 0x55dde2a682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 --2- 172.21.15.32:0/1849035250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5fff0 0x55dde2a59d50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 -- 172.21.15.32:0/1849035250 >> 172.21.15.32:0/1849035250 conn(0x55dde298c8f0 msgr2=0x55dde2a696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 -- 172.21.15.32:0/1849035250 shutdown_connections 2023-08-11T15:52:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.211+0000 7fcd42592b80 1 -- 172.21.15.32:0/1849035250 wait complete. 2023-08-11T15:52:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.212+0000 7fcd42592b80 1 -- start start 2023-08-11T15:52:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.212+0000 7fcd42592b80 1 Processor -- start 2023-08-11T15:52:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.212+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.212+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd42592b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2acbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd42592b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde2acc0f0 con 0x55dde2a64e80 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd42592b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde2acc260 con 0x55dde2a5f630 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd42592b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde2acc420 con 0x55dde2a5fff0 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd42592b80 10 client.0 _ll_register_callbacks cb 0x7fcd20009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a737b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:59176/0 (socket says 172.21.15.32:59176) 2023-08-11T15:52:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 -- 172.21.15.32:0/2727670587 learned_addr learned my addr 172.21.15.32:0/2727670587 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:52:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2acbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 -- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 msgr2=0x55dde2a73270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a73270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 -- 172.21.15.32:0/2727670587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 msgr2=0x55dde2acbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:52:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2acbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.213+0000 7fcd27fff700 1 -- 172.21.15.32:0/2727670587 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dde27ddd80 con 0x55dde2a5fff0 2023-08-11T15:52:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd27fff700 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a737b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcd14005920 tx=0x7fcd1401d550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f5f101c96bb5ac server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:52:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd07fff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:52:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd07fff700 1 -- 172.21.15.32:0/2727670587 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14016070 con 0x55dde2a5fff0 2023-08-11T15:52:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd07fff700 1 -- 172.21.15.32:0/2727670587 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fcd1401dac0 con 0x55dde2a5fff0 2023-08-11T15:52:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd2000f040 con 0x55dde2a5fff0 2023-08-11T15:52:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd07fff700 1 -- 172.21.15.32:0/2727670587 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd1401dc80 con 0x55dde2a5fff0 2023-08-11T15:52:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd42592b80 10 client.25897 fetch_fsmap learned FSMap version 29 2023-08-11T15:52:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.214+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fcd14004040 con 0x55dde2a5fff0 2023-08-11T15:52:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd07fff700 1 -- 172.21.15.32:0/2727670587 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2000f040 con 0x55dde2a5fff0 2023-08-11T15:52:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd07fff700 1 -- 172.21.15.32:0/2727670587 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd14004d00 con 0x55dde2a5fff0 2023-08-11T15:52:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd42592b80 10 client.25897 fetch_fsmap finished waiting for FSMap version 29 2023-08-11T15:52:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd42592b80 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:52:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd42592b80 20 client.25897 populate_metadata read hostname 'smithi032' 2023-08-11T15:52:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd42592b80 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2ad6300 0x55dde2ad8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:52:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd42592b80 4 client.25897 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:52:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dde29aa7c0 con 0x55dde2ad6300 2023-08-11T15:52:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.215+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2ad6300 0x55dde2ad8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:52:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.216+0000 7fcd2d19c700 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2ad6300 0x55dde2ad8720 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7032ebd67f35bf server_cookie=18ca89454eca99ac in_seq=0 out_seq=0 2023-08-11T15:52:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.216+0000 7fcd07fff700 10 client.25897 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd07fff700 1 -- 172.21.15.32:0/2727670587 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dde29aa7c0 con 0x55dde2ad6300 2023-08-11T15:52:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd07fff700 10 client.25897 handle_command_reply: tid=0 2023-08-11T15:52:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 client.25897 shutdown 2023-08-11T15:52:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 msgr2=0x55dde2a737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a737b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcd14005920 tx=0x7fcd1401d550 comp rx=0 tx=0).stop 2023-08-11T15:52:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 shutdown_connections 2023-08-11T15:52:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55dde2ad6300 0x55dde2ad8720 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dde2a64e80 0x55dde2acbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd07fff700 0 client.25897 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:52:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dde2a5f630 0x55dde2a73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 --2- 172.21.15.32:0/2727670587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dde2a5fff0 0x55dde2a737b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 >> 172.21.15.32:0/2727670587 conn(0x55dde298c8f0 msgr2=0x55dde2a6e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 shutdown_connections 2023-08-11T15:52:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 1 -- 172.21.15.32:0/2727670587 wait complete. 2023-08-11T15:52:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.523+0000 7fcd42592b80 20 client.25897 trim_cache size 0 max 16384 2023-08-11T15:52:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:52:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:52:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:52:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:52:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcd180a00c0 msgr2=0x7fcd180a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcd180a00c0 0x7fcd180a2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55dde2a5ae90 tx=0x7fcd1c022f90 comp rx=0 tx=0).stop 2023-08-11T15:52:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28102b00 msgr2=0x7fcd2819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:52:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28102b00 0x7fcd2819eb80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c005910 tx=0x7fcd1c005720 comp rx=0 tx=0).stop 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 shutdown_connections 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fcd180a00c0 0x7fcd180a2580 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd28108ba0 0x7fcd281989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28102b00 0x7fcd2819eb80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 --2- 172.21.15.32:0/874962092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd281034c0 0x7fcd2819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 >> 172.21.15.32:0/874962092 conn(0x7fcd280fe5d0 msgr2=0x7fcd280feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 shutdown_connections 2023-08-11T15:52:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:52:47.524+0000 7fcd2ebff700 1 -- 172.21.15.32:0/874962092 wait complete. 2023-08-11T15:52:47.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 692325 11.75 MB/sec execute 183 sec latency 196.856 ms 2023-08-11T15:52:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:47 smithi032 ceph-mon[97103]: pgmap v190: 129 pgs: 129 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 424 op/s 2023-08-11T15:52:48.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:47 smithi192 ceph-mon[130806]: pgmap v190: 129 pgs: 129 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 424 op/s 2023-08-11T15:52:48.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:47 smithi134 ceph-mon[130447]: pgmap v190: 129 pgs: 129 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 424 op/s 2023-08-11T15:52:48.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 692901 11.72 MB/sec execute 184 sec latency 91.969 ms 2023-08-11T15:52:49.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 694933 11.69 MB/sec execute 185 sec latency 49.838 ms 2023-08-11T15:52:50.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:50 smithi192 ceph-mon[130806]: pgmap v191: 129 pgs: 129 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 352 op/s 2023-08-11T15:52:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:50 smithi134 ceph-mon[130447]: pgmap v191: 129 pgs: 129 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 352 op/s 2023-08-11T15:52:50.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 696977 11.70 MB/sec execute 186 sec latency 46.815 ms 2023-08-11T15:52:50.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:50 smithi032 ceph-mon[97103]: pgmap v191: 129 pgs: 129 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 352 op/s 2023-08-11T15:52:51.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 699400 11.71 MB/sec execute 187 sec latency 99.939 ms 2023-08-11T15:52:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:52 smithi192 ceph-mon[130806]: pgmap v192: 129 pgs: 129 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 301 op/s 2023-08-11T15:52:52.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:52 smithi134 ceph-mon[130447]: pgmap v192: 129 pgs: 129 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 301 op/s 2023-08-11T15:52:52.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 701113 11.70 MB/sec execute 188 sec latency 27.436 ms 2023-08-11T15:52:52.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:52 smithi032 ceph-mon[97103]: pgmap v192: 129 pgs: 129 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 301 op/s 2023-08-11T15:52:53.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 703538 11.71 MB/sec execute 189 sec latency 16.214 ms 2023-08-11T15:52:54.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:54 smithi192 ceph-mon[130806]: pgmap v193: 129 pgs: 129 active+clean; 63 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 575 op/s 2023-08-11T15:52:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:54 smithi134 ceph-mon[130447]: pgmap v193: 129 pgs: 129 active+clean; 63 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 575 op/s 2023-08-11T15:52:54.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 705621 11.69 MB/sec execute 190 sec latency 15.443 ms 2023-08-11T15:52:54.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:54 smithi032 ceph-mon[97103]: pgmap v193: 129 pgs: 129 active+clean; 63 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 575 op/s 2023-08-11T15:52:55.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 707652 11.69 MB/sec execute 191 sec latency 17.438 ms 2023-08-11T15:52:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:56 smithi192 ceph-mon[130806]: pgmap v194: 129 pgs: 129 active+clean; 63 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 504 op/s 2023-08-11T15:52:56.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:56 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:52:56.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:56 smithi134 ceph-mon[130447]: pgmap v194: 129 pgs: 129 active+clean; 63 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 504 op/s 2023-08-11T15:52:56.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:56 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:52:56.763 INFO:tasks.workunit.client.0.smithi032.stdout: 1 710302 11.72 MB/sec execute 192 sec latency 49.731 ms 2023-08-11T15:52:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:56 smithi032 ceph-mon[97103]: pgmap v194: 129 pgs: 129 active+clean; 63 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 504 op/s 2023-08-11T15:52:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:56 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:52:57.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 711833 11.69 MB/sec execute 193 sec latency 60.118 ms 2023-08-11T15:52:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:57 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:52:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:57 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:52:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:57 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:52:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:52:57 smithi032 ceph-mon[97103]: pgmap v195: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 566 op/s 2023-08-11T15:52:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:57 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:52:58.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:57 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:52:58.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:57 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:52:58.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:52:57 smithi192 ceph-mon[130806]: pgmap v195: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 566 op/s 2023-08-11T15:52:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:57 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:52:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:57 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:52:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:57 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:52:58.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:52:57 smithi134 ceph-mon[130447]: pgmap v195: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 566 op/s 2023-08-11T15:52:58.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 714322 11.70 MB/sec execute 194 sec latency 11.687 ms 2023-08-11T15:52:59.763 INFO:tasks.workunit.client.0.smithi032.stdout: 1 716659 11.70 MB/sec execute 195 sec latency 51.072 ms 2023-08-11T15:53:00.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:00 smithi192 ceph-mon[130806]: pgmap v196: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 419 op/s 2023-08-11T15:53:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 -- 172.21.15.32:0/2635968998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78810d0f0 msgr2=0x7fa78810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 --2- 172.21.15.32:0/2635968998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78810d0f0 0x7fa78810f560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa780009f90 tx=0x7fa78000b330 comp rx=0 tx=0).stop 2023-08-11T15:53:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 -- 172.21.15.32:0/2635968998 shutdown_connections 2023-08-11T15:53:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 --2- 172.21.15.32:0/2635968998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78810d0f0 0x7fa78810f560 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 --2- 172.21.15.32:0/2635968998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa788072d50 0x7fa788073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 --2- 172.21.15.32:0/2635968998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788073670 0x7fa78810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 -- 172.21.15.32:0/2635968998 >> 172.21.15.32:0/2635968998 conn(0x7fa7880fc380 msgr2=0x7fa7880fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 -- 172.21.15.32:0/2635968998 shutdown_connections 2023-08-11T15:53:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.689+0000 7fa78ebff700 1 -- 172.21.15.32:0/2635968998 wait complete. 2023-08-11T15:53:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.690+0000 7fa78ebff700 1 Processor -- start 2023-08-11T15:53:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.690+0000 7fa78ebff700 1 -- start start 2023-08-11T15:53:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78ebff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa788072d50 0x7fa78819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78ebff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788073670 0x7fa78819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78ebff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa78810d0f0 0x7fa788198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78ebff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7881991c0 con 0x7fa788073670 2023-08-11T15:53:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78ebff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa788199330 con 0x7fa78810d0f0 2023-08-11T15:53:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78ebff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7881994f0 con 0x7fa788072d50 2023-08-11T15:53:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78c99b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa788072d50 0x7fa78819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa787fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788073670 0x7fa78819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78d19c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa78810d0f0 0x7fa788198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78c99b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa788072d50 0x7fa78819ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:49476/0 (socket says 172.21.15.32:49476) 2023-08-11T15:53:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78c99b700 1 -- 172.21.15.32:0/3165080970 learned_addr learned my addr 172.21.15.32:0/3165080970 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.691+0000 7fa78d19c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa78810d0f0 0x7fa788198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39600/0 (socket says 172.21.15.32:39600) 2023-08-11T15:53:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78c99b700 1 -- 172.21.15.32:0/3165080970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa78810d0f0 msgr2=0x7fa788198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78c99b700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa78810d0f0 0x7fa788198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78c99b700 1 -- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788073670 msgr2=0x7fa78819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78c99b700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788073670 0x7fa78819f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78c99b700 1 -- 172.21.15.32:0/3165080970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780009c40 con 0x7fa788072d50 2023-08-11T15:53:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa787fff700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788073670 0x7fa78819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78d19c700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa78810d0f0 0x7fa788198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78c99b700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa788072d50 0x7fa78819ecf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9c0 tx=0x7fa77c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa785ffb700 1 -- 172.21.15.32:0/3165080970 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c004a30 con 0x7fa788072d50 2023-08-11T15:53:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa785ffb700 1 -- 172.21.15.32:0/3165080970 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c00cc80 con 0x7fa788072d50 2023-08-11T15:53:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7881997d0 con 0x7fa788072d50 2023-08-11T15:53:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa785ffb700 1 -- 172.21.15.32:0/3165080970 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0054e0 con 0x7fa788072d50 2023-08-11T15:53:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.692+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7881a5ce0 con 0x7fa788072d50 2023-08-11T15:53:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.696+0000 7fa785ffb700 1 -- 172.21.15.32:0/3165080970 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa77c004b90 con 0x7fa788072d50 2023-08-11T15:53:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.697+0000 7fa7a2490b80 1 Processor -- start 2023-08-11T15:53:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa7a2490b80 1 -- start start 2023-08-11T15:53:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa785ffb700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa7780a0140 0x7fa7780a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa785ffb700 1 -- 172.21.15.32:0/3165080970 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa77c014070 con 0x7fa788072d50 2023-08-11T15:53:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa78d19c700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa7780a0140 0x7fa7780a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780630 0x55de11780ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de116d95d0 con 0x55de11785e80 2023-08-11T15:53:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de116d6ef0 con 0x55de11780630 2023-08-11T15:53:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.698+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de116d27e0 con 0x55de11780ff0 2023-08-11T15:53:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa78d19c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780630 0x55de11780ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa78c99b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780630 0x55de11780ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39614/0 (socket says 172.21.15.32:39614) 2023-08-11T15:53:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 -- 172.21.15.32:0/1084216540 learned_addr learned my addr 172.21.15.32:0/1084216540 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa78c99b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48958/0 (socket says 172.21.15.32:48958) 2023-08-11T15:53:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 -- 172.21.15.32:0/1084216540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 msgr2=0x55de1178cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 --2- 172.21.15.32:0/1084216540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 -- 172.21.15.32:0/1084216540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 msgr2=0x55de11786260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 --2- 172.21.15.32:0/1084216540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa787fff700 1 -- 172.21.15.32:0/1084216540 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de115525d0 con 0x55de11780630 2023-08-11T15:53:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.699+0000 7fa78d19c700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa7780a0140 0x7fa7780a2600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa780009550 tx=0x7fa780007ad0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.700+0000 7fa787fff700 1 --2- 172.21.15.32:0/1084216540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780630 0x55de11780ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7740114f0 tx=0x7fa774011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d501141d1fc1f93 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.700+0000 7fa772ffd700 1 -- 172.21.15.32:0/1084216540 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774004030 con 0x55de11780630 2023-08-11T15:53:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.700+0000 7fa772ffd700 1 -- 172.21.15.32:0/1084216540 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa774003680 con 0x55de11780630 2023-08-11T15:53:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.700+0000 7fa772ffd700 1 -- 172.21.15.32:0/1084216540 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77400e6d0 con 0x55de11780630 2023-08-11T15:53:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 -- 172.21.15.32:0/1084216540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780630 msgr2=0x55de11780ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/1084216540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780630 0x55de11780ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7740114f0 tx=0x7fa774011800 comp rx=0 tx=0).stop 2023-08-11T15:53:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 -- 172.21.15.32:0/1084216540 shutdown_connections 2023-08-11T15:53:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/1084216540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/1084216540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/1084216540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780630 0x55de11780ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 -- 172.21.15.32:0/1084216540 >> 172.21.15.32:0/1084216540 conn(0x55de116ad8f0 msgr2=0x55de1177ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 -- 172.21.15.32:0/1084216540 shutdown_connections 2023-08-11T15:53:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.701+0000 7fa7a2490b80 1 -- 172.21.15.32:0/1084216540 wait complete. 2023-08-11T15:53:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.702+0000 7fa7a2490b80 1 -- start start 2023-08-11T15:53:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.702+0000 7fa7a2490b80 1 Processor -- start 2023-08-11T15:53:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11785e80 0x55de11789ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11670a60 con 0x55de11780630 2023-08-11T15:53:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11670760 con 0x55de11785e80 2023-08-11T15:53:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de115548c0 con 0x55de11780ff0 2023-08-11T15:53:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa7a2490b80 10 client.0 _ll_register_callbacks cb 0x55de11673a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa78d19c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11785e80 0x55de11789ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa78d19c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11785e80 0x55de11789ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39624/0 (socket says 172.21.15.32:39624) 2023-08-11T15:53:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa78d19c700 1 -- 172.21.15.32:0/3226062039 learned_addr learned my addr 172.21.15.32:0/3226062039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa78c99b700 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.703+0000 7fa787fff700 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa78d19c700 1 -- 172.21.15.32:0/3226062039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 msgr2=0x55de1178fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa78d19c700 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa78d19c700 1 -- 172.21.15.32:0/3226062039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 msgr2=0x55de1178f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa78d19c700 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa78d19c700 1 -- 172.21.15.32:0/3226062039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa774009cd0 con 0x55de11785e80 2023-08-11T15:53:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa78d19c700 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11785e80 0x55de11789ce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa780015000 tx=0x7fa78000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc482ec1dcefb3b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa767fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.704+0000 7fa767fff700 1 -- 172.21.15.32:0/3226062039 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780025030 con 0x55de11785e80 2023-08-11T15:53:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de117737e0 con 0x55de11785e80 2023-08-11T15:53:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa767fff700 1 -- 172.21.15.32:0/3226062039 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa780026070 con 0x55de11785e80 2023-08-11T15:53:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa767fff700 1 -- 172.21.15.32:0/3226062039 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa78000e040 con 0x55de11785e80 2023-08-11T15:53:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa7a2490b80 10 client.25907 fetch_fsmap learned FSMap version 29 2023-08-11T15:53:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55de114fed80 con 0x55de11785e80 2023-08-11T15:53:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa767fff700 1 -- 172.21.15.32:0/3226062039 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de117737e0 con 0x55de11785e80 2023-08-11T15:53:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa767fff700 1 -- 172.21.15.32:0/3226062039 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa78002aae0 con 0x55de11785e80 2023-08-11T15:53:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa7a2490b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 29 2023-08-11T15:53:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.705+0000 7fa7a2490b80 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.706+0000 7fa7a2490b80 20 client.25907 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.706+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.706+0000 7fa7a2490b80 4 client.25907 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:53:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.706+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de1174f480 con 0x55de117a2670 2023-08-11T15:53:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.706+0000 7fa78c99b700 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.706+0000 7fa78c99b700 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117a4a90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73f304ccdff2d02 server_cookie=8baede505c51cf98 in_seq=0 out_seq=0 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.706+0000 7fa767fff700 10 client.25907 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa767fff700 1 -- 172.21.15.32:0/3226062039 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55de1174f480 con 0x55de117a2670 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa767fff700 10 client.25907 handle_command_reply: tid=0 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa7a2490b80 1 client.25907 shutdown 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11785e80 msgr2=0x55de11789ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11785e80 0x55de11789ce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa780015000 tx=0x7fa78000b330 comp rx=0 tx=0).stop 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 shutdown_connections 2023-08-11T15:53:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117a4a90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.707+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.708+0000 7fa767fff700 0 client.25907 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.708+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11785e80 0x55de11789ce0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.708+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/3226062039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780ff0 0x55de1178fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.708+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 >> 172.21.15.32:0/3226062039 conn(0x55de116ad8f0 msgr2=0x55de1177a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.708+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 shutdown_connections 2023-08-11T15:53:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.708+0000 7fa7a2490b80 1 -- 172.21.15.32:0/3226062039 wait complete. 2023-08-11T15:53:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.708+0000 7fa7a2490b80 20 client.25907 trim_cache size 0 max 16384 2023-08-11T15:53:00.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:00 smithi134 ceph-mon[130447]: pgmap v196: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 419 op/s 2023-08-11T15:53:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.730+0000 7fa7a2490b80 1 Processor -- start 2023-08-11T15:53:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.730+0000 7fa7a2490b80 1 -- start start 2023-08-11T15:53:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780630 0x55de1177ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de117869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1178b030 con 0x55de11785e80 2023-08-11T15:53:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11784a40 con 0x55de11780ff0 2023-08-11T15:53:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1179ef90 con 0x55de11780630 2023-08-11T15:53:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa787fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de117869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780630 0x55de1177ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780630 0x55de1177ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:49506/0 (socket says 172.21.15.32:49506) 2023-08-11T15:53:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 -- 172.21.15.32:0/4276470583 learned_addr learned my addr 172.21.15.32:0/4276470583 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa787fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de117869f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39630/0 (socket says 172.21.15.32:39630) 2023-08-11T15:53:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78d19c700 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 -- 172.21.15.32:0/4276470583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 msgr2=0x55de117869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de117869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 -- 172.21.15.32:0/4276470583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 msgr2=0x55de11786f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.731+0000 7fa78c99b700 1 -- 172.21.15.32:0/4276470583 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de114e7c00 con 0x55de11780630 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.732+0000 7fa78c99b700 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780630 0x55de1177ad50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0957e0 tx=0x7fa77c098c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d696118fd261303a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.732+0000 7fa7737fe700 1 -- 172.21.15.32:0/4276470583 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0a0b80 con 0x55de11780630 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.732+0000 7fa7737fe700 1 -- 172.21.15.32:0/4276470583 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c097110 con 0x55de11780630 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.732+0000 7fa7737fe700 1 -- 172.21.15.32:0/4276470583 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c016330 con 0x55de11780630 2023-08-11T15:53:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 -- 172.21.15.32:0/4276470583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780630 msgr2=0x55de1177ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780630 0x55de1177ad50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0957e0 tx=0x7fa77c098c70 comp rx=0 tx=0).stop 2023-08-11T15:53:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 -- 172.21.15.32:0/4276470583 shutdown_connections 2023-08-11T15:53:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11785e80 0x55de11786f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11780630 0x55de1177ad50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/4276470583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de117869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 -- 172.21.15.32:0/4276470583 >> 172.21.15.32:0/4276470583 conn(0x55de116ad8f0 msgr2=0x55de11783f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 -- 172.21.15.32:0/4276470583 shutdown_connections 2023-08-11T15:53:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.733+0000 7fa7a2490b80 1 -- 172.21.15.32:0/4276470583 wait complete. 2023-08-11T15:53:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.734+0000 7fa7a2490b80 1 -- start start 2023-08-11T15:53:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.734+0000 7fa7a2490b80 1 Processor -- start 2023-08-11T15:53:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de1178d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa7a2490b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11785e80 0x55de1178daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1178e070 con 0x55de11780630 2023-08-11T15:53:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1178e1b0 con 0x55de11780ff0 2023-08-11T15:53:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa7a2490b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1178e370 con 0x55de11785e80 2023-08-11T15:53:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa7a2490b80 10 client.0 _ll_register_callbacks cb 0x55de11673a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa787fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de1178d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa787fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de1178d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:39640/0 (socket says 172.21.15.32:39640) 2023-08-11T15:53:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa787fff700 1 -- 172.21.15.32:0/936430294 learned_addr learned my addr 172.21.15.32:0/936430294 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa78d19c700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11785e80 0x55de1178daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa78c99b700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa787fff700 1 -- 172.21.15.32:0/936430294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11785e80 msgr2=0x55de1178daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa787fff700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11785e80 0x55de1178daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa787fff700 1 -- 172.21.15.32:0/936430294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 msgr2=0x55de1178f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.735+0000 7fa787fff700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa787fff700 1 -- 172.21.15.32:0/936430294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7881a5ce0 con 0x55de11780ff0 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa78d19c700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11785e80 0x55de1178daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa78c99b700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa787fff700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de1178d5b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7740097c0 tx=0x7fa774005390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1b27721d08feb5b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa767fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa767fff700 1 -- 172.21.15.32:0/936430294 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774005ce0 con 0x55de11780ff0 2023-08-11T15:53:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa774003680 con 0x55de11780ff0 2023-08-11T15:53:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa767fff700 1 -- 172.21.15.32:0/936430294 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa77402a430 con 0x55de11780ff0 2023-08-11T15:53:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.736+0000 7fa767fff700 1 -- 172.21.15.32:0/936430294 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa77402a5f0 con 0x55de11780ff0 2023-08-11T15:53:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 10 client.25913 fetch_fsmap learned FSMap version 29 2023-08-11T15:53:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55de115251f0 con 0x55de11780ff0 2023-08-11T15:53:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa767fff700 1 -- 172.21.15.32:0/936430294 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa774003680 con 0x55de11780ff0 2023-08-11T15:53:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa767fff700 1 -- 172.21.15.32:0/936430294 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa7740195b0 con 0x55de11780ff0 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 29 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 20 client.25913 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117f9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 4 client.25913 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de116cb7c0 con 0x55de117a2670 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.737+0000 7fa78c99b700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.738+0000 7fa78c99b700 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117f9850 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cac7ccbedbad2cf server_cookie=3c327c93bc587fd4 in_seq=0 out_seq=0 2023-08-11T15:53:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:00.738+0000 7fa767fff700 10 client.25913 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:00.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 718520 11.69 MB/sec execute 196 sec latency 32.243 ms 2023-08-11T15:53:00.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:00 smithi032 ceph-mon[97103]: pgmap v196: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 419 op/s 2023-08-11T15:53:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.012+0000 7fa767fff700 1 -- 172.21.15.32:0/936430294 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de116cb7c0 con 0x55de117a2670 2023-08-11T15:53:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.012+0000 7fa767fff700 10 client.25913 handle_command_reply: tid=0 2023-08-11T15:53:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.012+0000 7fa7a2490b80 1 client.25913 shutdown 2023-08-11T15:53:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 msgr2=0x55de1178d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de1178d5b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7740097c0 tx=0x7fa774005390 comp rx=0 tx=0).stop 2023-08-11T15:53:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 shutdown_connections 2023-08-11T15:53:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55de117a2670 0x55de117f9850 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55de11780630 0x55de1178f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de11785e80 0x55de1178daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 --2- 172.21.15.32:0/936430294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de11780ff0 0x55de1178d5b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 >> 172.21.15.32:0/936430294 conn(0x55de116ad8f0 msgr2=0x55de1177f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa767fff700 0 client.25913 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 shutdown_connections 2023-08-11T15:53:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.013+0000 7fa7a2490b80 1 -- 172.21.15.32:0/936430294 wait complete. 2023-08-11T15:53:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.014+0000 7fa7a2490b80 20 client.25913 trim_cache size 0 max 16384 2023-08-11T15:53:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:53:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:53:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:53:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:53:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.014+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa7780a0140 msgr2=0x7fa7780a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.014+0000 7fa78ebff700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa7780a0140 0x7fa7780a2600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa780009550 tx=0x7fa780007ad0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.014+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa788072d50 msgr2=0x7fa78819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.014+0000 7fa78ebff700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa788072d50 0x7fa78819ecf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9c0 tx=0x7fa77c00ecd0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 shutdown_connections 2023-08-11T15:53:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa7780a0140 0x7fa7780a2600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa78810d0f0 0x7fa788198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa788072d50 0x7fa78819ecf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 --2- 172.21.15.32:0/3165080970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788073670 0x7fa78819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 >> 172.21.15.32:0/3165080970 conn(0x7fa7880fc380 msgr2=0x7fa78810f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 shutdown_connections 2023-08-11T15:53:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:01.015+0000 7fa78ebff700 1 -- 172.21.15.32:0/3165080970 wait complete. 2023-08-11T15:53:01.763 INFO:tasks.workunit.client.0.smithi032.stdout: 1 719597 11.65 MB/sec execute 197 sec latency 148.369 ms 2023-08-11T15:53:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:02 smithi192 ceph-mon[130806]: pgmap v197: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2023-08-11T15:53:02.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:02 smithi134 ceph-mon[130447]: pgmap v197: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2023-08-11T15:53:02.763 INFO:tasks.workunit.client.0.smithi032.stdout: 1 721100 11.66 MB/sec execute 198 sec latency 61.282 ms 2023-08-11T15:53:02.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:02 smithi032 ceph-mon[97103]: pgmap v197: 129 pgs: 129 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2023-08-11T15:53:03.763 INFO:tasks.workunit.client.0.smithi032.stdout: 1 721591 11.60 MB/sec execute 199 sec latency 53.953 ms 2023-08-11T15:53:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:03 smithi032 ceph-mon[97103]: osdmap e98: 12 total, 12 up, 12 in 2023-08-11T15:53:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:03 smithi032 ceph-mon[97103]: pgmap v199: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2023-08-11T15:53:04.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:03 smithi192 ceph-mon[130806]: osdmap e98: 12 total, 12 up, 12 in 2023-08-11T15:53:04.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:03 smithi192 ceph-mon[130806]: pgmap v199: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2023-08-11T15:53:04.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:03 smithi134 ceph-mon[130447]: osdmap e98: 12 total, 12 up, 12 in 2023-08-11T15:53:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:03 smithi134 ceph-mon[130447]: pgmap v199: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2023-08-11T15:53:04.764 INFO:tasks.workunit.client.0.smithi032.stdout: 1 721846 11.55 MB/sec execute 200 sec latency 65.413 ms 2023-08-11T15:53:05.764 INFO:tasks.workunit.client.0.smithi032.stdout: 1 722496 11.51 MB/sec execute 201 sec latency 71.357 ms 2023-08-11T15:53:06.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:06 smithi134 ceph-mon[130447]: pgmap v200: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2023-08-11T15:53:06.765 INFO:tasks.workunit.client.0.smithi032.stdout: 1 723286 11.46 MB/sec execute 202 sec latency 47.091 ms 2023-08-11T15:53:06.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:06 smithi032 ceph-mon[97103]: pgmap v200: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2023-08-11T15:53:06.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:06 smithi192 ceph-mon[130806]: pgmap v200: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2023-08-11T15:53:07.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 724473 11.46 MB/sec execute 203 sec latency 43.751 ms 2023-08-11T15:53:08.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:07 smithi032 ceph-mon[97103]: pgmap v201: 129 pgs: 6 active+clean+snaptrim, 110 active+clean, 13 active+clean+snaptrim_wait; 61 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.1 MiB/s wr, 263 op/s 2023-08-11T15:53:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:07 smithi192 ceph-mon[130806]: pgmap v201: 129 pgs: 6 active+clean+snaptrim, 110 active+clean, 13 active+clean+snaptrim_wait; 61 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.1 MiB/s wr, 263 op/s 2023-08-11T15:53:08.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:07 smithi134 ceph-mon[130447]: pgmap v201: 129 pgs: 6 active+clean+snaptrim, 110 active+clean, 13 active+clean+snaptrim_wait; 61 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.1 MiB/s wr, 263 op/s 2023-08-11T15:53:08.765 INFO:tasks.workunit.client.0.smithi032.stdout: 1 725194 11.41 MB/sec execute 204 sec latency 123.221 ms 2023-08-11T15:53:09.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 726099 11.38 MB/sec execute 205 sec latency 290.398 ms 2023-08-11T15:53:10.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:10 smithi192 ceph-mon[130806]: pgmap v202: 129 pgs: 8 active+clean+snaptrim, 103 active+clean, 18 active+clean+snaptrim_wait; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2023-08-11T15:53:10.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:10 smithi134 ceph-mon[130447]: pgmap v202: 129 pgs: 8 active+clean+snaptrim, 103 active+clean, 18 active+clean+snaptrim_wait; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2023-08-11T15:53:10.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 728329 11.39 MB/sec execute 206 sec latency 52.592 ms 2023-08-11T15:53:10.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:10 smithi032 ceph-mon[97103]: pgmap v202: 129 pgs: 8 active+clean+snaptrim, 103 active+clean, 18 active+clean+snaptrim_wait; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2023-08-11T15:53:11.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 730500 11.38 MB/sec execute 207 sec latency 19.904 ms 2023-08-11T15:53:12.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:12 smithi192 ceph-mon[130806]: pgmap v203: 129 pgs: 8 active+clean+snaptrim, 103 active+clean, 18 active+clean+snaptrim_wait; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2023-08-11T15:53:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:12 smithi134 ceph-mon[130447]: pgmap v203: 129 pgs: 8 active+clean+snaptrim, 103 active+clean, 18 active+clean+snaptrim_wait; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2023-08-11T15:53:12.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 732488 11.38 MB/sec execute 208 sec latency 19.296 ms 2023-08-11T15:53:12.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:12 smithi032 ceph-mon[97103]: pgmap v203: 129 pgs: 8 active+clean+snaptrim, 103 active+clean, 18 active+clean+snaptrim_wait; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2023-08-11T15:53:13.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 734516 11.38 MB/sec execute 209 sec latency 56.799 ms 2023-08-11T15:53:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 -- 172.21.15.32:0/944136699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e01068c0 msgr2=0x7f64e0106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 --2- 172.21.15.32:0/944136699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e01068c0 0x7f64e0106ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009990 tx=0x7f64dc009ca0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 -- 172.21.15.32:0/944136699 shutdown_connections 2023-08-11T15:53:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 --2- 172.21.15.32:0/944136699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64e0101270 0x7f64e010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 --2- 172.21.15.32:0/944136699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e01068c0 0x7f64e0106ca0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 --2- 172.21.15.32:0/944136699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64e01008b0 0x7f64e0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 -- 172.21.15.32:0/944136699 >> 172.21.15.32:0/944136699 conn(0x7f64e00fc380 msgr2=0x7f64e00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 -- 172.21.15.32:0/944136699 shutdown_connections 2023-08-11T15:53:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.171+0000 7f64e7dff700 1 -- 172.21.15.32:0/944136699 wait complete. 2023-08-11T15:53:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.173+0000 7f64e7dff700 1 Processor -- start 2023-08-11T15:53:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.173+0000 7f64e7dff700 1 -- start start 2023-08-11T15:53:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e7dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64e01008b0 0x7f64e0072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e7dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0101270 0x7f64e0073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e7dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64e01068c0 0x7f64e01a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e7dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01a50a0 con 0x7f64e01008b0 2023-08-11T15:53:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e7dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01a51e0 con 0x7f64e0101270 2023-08-11T15:53:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e7dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01a5350 con 0x7f64e01068c0 2023-08-11T15:53:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0101270 0x7f64e0073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64e01008b0 0x7f64e0072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0101270 0x7f64e0073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:60996/0 (socket says 172.21.15.32:60996) 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 -- 172.21.15.32:0/713805455 learned_addr learned my addr 172.21.15.32:0/713805455 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e639c700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64e01068c0 0x7f64e01a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 -- 172.21.15.32:0/713805455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64e01068c0 msgr2=0x7f64e01a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64e01068c0 0x7f64e01a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 -- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64e01008b0 msgr2=0x7f64e0072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64e01008b0 0x7f64e0072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 -- 172.21.15.32:0/713805455 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0095f0 con 0x7f64e0101270 2023-08-11T15:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.174+0000 7f64e539a700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0101270 0x7f64e0073340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e9a0 tx=0x7f64d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.175+0000 7f64d6ffd700 1 -- 172.21.15.32:0/713805455 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0009790 con 0x7f64e0101270 2023-08-11T15:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.175+0000 7f64d6ffd700 1 -- 172.21.15.32:0/713805455 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d0010430 con 0x7f64e0101270 2023-08-11T15:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.175+0000 7f64d6ffd700 1 -- 172.21.15.32:0/713805455 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0018680 con 0x7f64e0101270 2023-08-11T15:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.175+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e01a5630 con 0x7f64e0101270 2023-08-11T15:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.175+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e01a5b40 con 0x7f64e0101270 2023-08-11T15:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.178+0000 7f64d6ffd700 1 -- 172.21.15.32:0/713805455 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f64d00105a0 con 0x7f64e0101270 2023-08-11T15:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.179+0000 7f64d6ffd700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64cc0a0140 0x7f64cc0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.179+0000 7f64fb799b80 1 Processor -- start 2023-08-11T15:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64d6ffd700 1 -- 172.21.15.32:0/713805455 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f64d0014070 con 0x7f64e0101270 2023-08-11T15:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64fb799b80 1 -- start start 2023-08-11T15:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64cc0a0140 0x7f64cc0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb63c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb63d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64fb799b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb63205d0 con 0x563bb63c7630 2023-08-11T15:53:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64fb799b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb631def0 con 0x563bb63c7ff0 2023-08-11T15:53:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64fb799b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb63197e0 con 0x563bb63cce80 2023-08-11T15:53:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64e539a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb63c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64e639c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb63d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:47216/0 (socket says 172.21.15.32:47216) 2023-08-11T15:53:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64e5b9b700 1 -- 172.21.15.32:0/1471815004 learned_addr learned my addr 172.21.15.32:0/1471815004 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.180+0000 7f64e639c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb63d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:32776/0 (socket says 172.21.15.32:32776) 2023-08-11T15:53:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e639c700 1 -- 172.21.15.32:0/1471815004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 msgr2=0x563bb63cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e639c700 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e639c700 1 -- 172.21.15.32:0/1471815004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 msgr2=0x563bb63c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e639c700 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb63c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e639c700 1 -- 172.21.15.32:0/1471815004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb61995d0 con 0x563bb63c7ff0 2023-08-11T15:53:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e539a700 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb63c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64cc0a0140 0x7f64cc0a2600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64dc015000 tx=0x7f64dc00bb20 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64e639c700 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb63d3c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64d8011580 tx=0x7f64d8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1884499ac81b7e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64cbfff700 1 -- 172.21.15.32:0/1471815004 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d800e040 con 0x563bb63c7ff0 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64cbfff700 1 -- 172.21.15.32:0/1471815004 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d800f040 con 0x563bb63c7ff0 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.181+0000 7f64cbfff700 1 -- 172.21.15.32:0/1471815004 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8004300 con 0x563bb63c7ff0 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 -- 172.21.15.32:0/1471815004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 msgr2=0x563bb63d3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb63d3c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64d8011580 tx=0x7f64d8009920 comp rx=0 tx=0).stop 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 -- 172.21.15.32:0/1471815004 shutdown_connections 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb63d3c10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1471815004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb63c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 -- 172.21.15.32:0/1471815004 >> 172.21.15.32:0/1471815004 conn(0x563bb62f48f0 msgr2=0x563bb63c1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 -- 172.21.15.32:0/1471815004 shutdown_connections 2023-08-11T15:53:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 -- 172.21.15.32:0/1471815004 wait complete. 2023-08-11T15:53:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.182+0000 7f64fb799b80 1 -- start start 2023-08-11T15:53:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 1 Processor -- start 2023-08-11T15:53:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63c7ff0 0x563bb63d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63cce80 0x563bb63d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb62b7a60 con 0x563bb63cce80 2023-08-11T15:53:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb62b7760 con 0x563bb63c7630 2023-08-11T15:53:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb619b8c0 con 0x563bb63c7ff0 2023-08-11T15:53:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.183+0000 7f64fb799b80 10 client.0 _ll_register_callbacks cb 0x563bb62baa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:32788/0 (socket says 172.21.15.32:32788) 2023-08-11T15:53:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 -- 172.21.15.32:0/3716069295 learned_addr learned my addr 172.21.15.32:0/3716069295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e539a700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63c7ff0 0x563bb63d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e639c700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63cce80 0x563bb63d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 -- 172.21.15.32:0/3716069295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63c7ff0 msgr2=0x563bb63d6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63c7ff0 0x563bb63d6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 -- 172.21.15.32:0/3716069295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63cce80 msgr2=0x563bb63d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63cce80 0x563bb63d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 -- 172.21.15.32:0/3716069295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e01a5b40 con 0x563bb63c7630 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e539a700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63c7ff0 0x563bb63d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d6990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b2e0 tx=0x7f64dc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b6b221dbf9a74f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.184+0000 7f64c97fa700 1 -- 172.21.15.32:0/3716069295 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc004990 con 0x563bb63c7630 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bb63ba7e0 con 0x563bb63c7630 2023-08-11T15:53:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64c97fa700 1 -- 172.21.15.32:0/3716069295 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f64dc004af0 con 0x563bb63c7630 2023-08-11T15:53:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64c97fa700 1 -- 172.21.15.32:0/3716069295 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64dc004cb0 con 0x563bb63c7630 2023-08-11T15:53:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 10 client.25925 fetch_fsmap learned FSMap version 29 2023-08-11T15:53:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f64d8011230 con 0x563bb63c7630 2023-08-11T15:53:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64c97fa700 1 -- 172.21.15.32:0/3716069295 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bb63ba7e0 con 0x563bb63c7630 2023-08-11T15:53:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64c97fa700 1 -- 172.21.15.32:0/3716069295 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64dc018400 con 0x563bb63c7630 2023-08-11T15:53:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 10 client.25925 fetch_fsmap finished waiting for FSMap version 29 2023-08-11T15:53:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 20 client.25925 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb63e0d40 0x563bb63e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 4 client.25925 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:53:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.185+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bb6396480 con 0x563bb63e0d40 2023-08-11T15:53:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.186+0000 7f64e539a700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb63e0d40 0x563bb63e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.186+0000 7f64e539a700 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb63e0d40 0x563bb63e3160 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e49de512e27a49 server_cookie=4e7ef532a82d0f36 in_seq=0 out_seq=0 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.186+0000 7f64c97fa700 10 client.25925 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64c97fa700 1 -- 172.21.15.32:0/3716069295 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563bb6396480 con 0x563bb63e0d40 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64c97fa700 10 client.25925 handle_command_reply: tid=0 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 client.25925 shutdown 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 msgr2=0x563bb63d6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d6990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b2e0 tx=0x7f64dc009ca0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 shutdown_connections 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb63e0d40 0x563bb63e3160 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63cce80 0x563bb63d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64c97fa700 0 client.25925 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d6990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3716069295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63c7ff0 0x563bb63d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 >> 172.21.15.32:0/3716069295 conn(0x563bb62f48f0 msgr2=0x563bb63c15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.187+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 shutdown_connections 2023-08-11T15:53:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.188+0000 7f64fb799b80 1 -- 172.21.15.32:0/3716069295 wait complete. 2023-08-11T15:53:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.188+0000 7f64fb799b80 20 client.25925 trim_cache size 0 max 16384 2023-08-11T15:53:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 Processor -- start 2023-08-11T15:53:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 -- start start 2023-08-11T15:53:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7ff0 0x563bb63cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb63c3db0 con 0x563bb63c7ff0 2023-08-11T15:53:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb63cb9a0 con 0x563bb63c7630 2023-08-11T15:53:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.203+0000 7f64fb799b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb63c83d0 con 0x563bb63cce80 2023-08-11T15:53:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e539a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7ff0 0x563bb63cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:32790/0 (socket says 172.21.15.32:32790) 2023-08-11T15:53:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 -- 172.21.15.32:0/1365555725 learned_addr learned my addr 172.21.15.32:0/1365555725 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e639c700 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 -- 172.21.15.32:0/1365555725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 msgr2=0x563bb63c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 -- 172.21.15.32:0/1365555725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7ff0 msgr2=0x563bb63cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7ff0 0x563bb63cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 -- 172.21.15.32:0/1365555725 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb6145d80 con 0x563bb63c7630 2023-08-11T15:53:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.204+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d64f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b6f0 tx=0x7f64dc0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63d04f2c72f43e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64d4bff700 1 -- 172.21.15.32:0/1365555725 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc018400 con 0x563bb63c7630 2023-08-11T15:53:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64d4bff700 1 -- 172.21.15.32:0/1365555725 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64dc01e040 con 0x563bb63c7630 2023-08-11T15:53:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64d4bff700 1 -- 172.21.15.32:0/1365555725 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00f460 con 0x563bb63c7630 2023-08-11T15:53:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 -- 172.21.15.32:0/1365555725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 msgr2=0x563bb63d64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d64f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b6f0 tx=0x7f64dc0057a0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 -- 172.21.15.32:0/1365555725 shutdown_connections 2023-08-11T15:53:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb63c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7630 0x563bb63d64f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 --2- 172.21.15.32:0/1365555725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7ff0 0x563bb63cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 -- 172.21.15.32:0/1365555725 >> 172.21.15.32:0/1365555725 conn(0x563bb62f48f0 msgr2=0x563bb63cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 -- 172.21.15.32:0/1365555725 shutdown_connections 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.205+0000 7f64fb799b80 1 -- 172.21.15.32:0/1365555725 wait complete. 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.206+0000 7f64fb799b80 1 -- start start 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.206+0000 7f64fb799b80 1 Processor -- start 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.206+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb6428b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.206+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb64290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64fb799b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb642da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64fb799b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb642df80 con 0x563bb63c7630 2023-08-11T15:53:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64fb799b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb642e0f0 con 0x563bb63c7ff0 2023-08-11T15:53:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64fb799b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb642e2b0 con 0x563bb63cce80 2023-08-11T15:53:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64fb799b80 10 client.0 _ll_register_callbacks cb 0x563bb62baa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb64290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e5b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb6428b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb64290b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:32798/0 (socket says 172.21.15.32:32798) 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e539a700 1 -- 172.21.15.32:0/3786978527 learned_addr learned my addr 172.21.15.32:0/3786978527 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e639c700 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb642da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e5b9b700 1 -- 172.21.15.32:0/3786978527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 msgr2=0x563bb642da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb642da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e5b9b700 1 -- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 msgr2=0x563bb64290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb64290b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e5b9b700 1 -- 172.21.15.32:0/3786978527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d8011230 con 0x563bb63c7630 2023-08-11T15:53:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e539a700 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb64290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64e5b9b700 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb6428b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b2e0 tx=0x7f64dc018690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4620bf3807523cb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.207+0000 7f64c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:53:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64c97fa700 1 -- 172.21.15.32:0/3786978527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc018a90 con 0x563bb63c7630 2023-08-11T15:53:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64d800f040 con 0x563bb63c7630 2023-08-11T15:53:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64c97fa700 1 -- 172.21.15.32:0/3786978527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f64dc0049a0 con 0x563bb63c7630 2023-08-11T15:53:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64c97fa700 1 -- 172.21.15.32:0/3786978527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64dc004b60 con 0x563bb63c7630 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64fb799b80 10 client.15915 fetch_fsmap learned FSMap version 29 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563bb612ec00 con 0x563bb63c7630 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64c97fa700 1 -- 172.21.15.32:0/3786978527 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d800f040 con 0x563bb63c7630 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64c97fa700 1 -- 172.21.15.32:0/3786978527 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64dc038040 con 0x563bb63c7630 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.208+0000 7f64fb799b80 10 client.15915 fetch_fsmap finished waiting for FSMap version 29 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64fb799b80 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64fb799b80 20 client.15915 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb6435660 0x563bb6437a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64fb799b80 4 client.15915 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:53:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563bb63127c0 con 0x563bb6435660 2023-08-11T15:53:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64e539a700 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb6435660 0x563bb6437a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64e539a700 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb6435660 0x563bb6437a50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d8722da6fee569 server_cookie=3366befcef6f6428 in_seq=0 out_seq=0 2023-08-11T15:53:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.209+0000 7f64c97fa700 10 client.15915 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.384+0000 7f64c97fa700 1 -- 172.21.15.32:0/3786978527 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563bb63127c0 con 0x563bb6435660 2023-08-11T15:53:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.384+0000 7f64c97fa700 10 client.15915 handle_command_reply: tid=0 2023-08-11T15:53:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.384+0000 7f64fb799b80 1 client.15915 shutdown 2023-08-11T15:53:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.384+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 msgr2=0x563bb6428b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.384+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb6428b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b2e0 tx=0x7f64dc018690 comp rx=0 tx=0).stop 2023-08-11T15:53:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 shutdown_connections 2023-08-11T15:53:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x563bb6435660 0x563bb6437a50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563bb63c7630 0x563bb6428b70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bb63cce80 0x563bb642da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 --2- 172.21.15.32:0/3786978527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb63c7ff0 0x563bb64290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 >> 172.21.15.32:0/3786978527 conn(0x563bb62f48f0 msgr2=0x563bb63c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64c97fa700 0 client.15915 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 shutdown_connections 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 1 -- 172.21.15.32:0/3786978527 wait complete. 2023-08-11T15:53:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.385+0000 7f64fb799b80 20 client.15915 trim_cache size 0 max 16384 2023-08-11T15:53:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:53:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:53:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:53:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:53:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.386+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64cc0a0140 msgr2=0x7f64cc0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.386+0000 7f64e7dff700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64cc0a0140 0x7f64cc0a2600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64dc015000 tx=0x7f64dc00bb20 comp rx=0 tx=0).stop 2023-08-11T15:53:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.386+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0101270 msgr2=0x7f64e0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.386+0000 7f64e7dff700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0101270 0x7f64e0073340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e9a0 tx=0x7f64d000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 shutdown_connections 2023-08-11T15:53:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f64cc0a0140 0x7f64cc0a2600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64e01008b0 0x7f64e0072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64e01068c0 0x7f64e01a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 --2- 172.21.15.32:0/713805455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0101270 0x7f64e0073340 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 >> 172.21.15.32:0/713805455 conn(0x7f64e00fc380 msgr2=0x7f64e01043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 shutdown_connections 2023-08-11T15:53:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:14.387+0000 7f64e7dff700 1 -- 172.21.15.32:0/713805455 wait complete. 2023-08-11T15:53:14.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:14 smithi134 ceph-mon[130447]: pgmap v204: 129 pgs: 129 active+clean; 68 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 4.5 MiB/s wr, 251 op/s 2023-08-11T15:53:14.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 736160 11.37 MB/sec execute 210 sec latency 97.506 ms 2023-08-11T15:53:14.772 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:14 smithi192 ceph-mon[130806]: pgmap v204: 129 pgs: 129 active+clean; 68 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 4.5 MiB/s wr, 251 op/s 2023-08-11T15:53:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:14 smithi032 ceph-mon[97103]: pgmap v204: 129 pgs: 129 active+clean; 68 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 4.5 MiB/s wr, 251 op/s 2023-08-11T15:53:15.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:15 smithi134 ceph-mon[130447]: daemon mds.h restarted 2023-08-11T15:53:15.471 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:53:15 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: ignoring --setuser ceph since I am not root 2023-08-11T15:53:15.471 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:53:15 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: ignoring --setgroup ceph since I am not root 2023-08-11T15:53:15.471 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:53:15 smithi134 ceph-mds[155206]: -- 172.21.15.134:0/2150217212 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d56db511e0 con 0x55d56db26800 2023-08-11T15:53:15.472 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:53:15 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: starting mds.h at 2023-08-11T15:53:15.722 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:15 smithi192 ceph-mon[130806]: daemon mds.h restarted 2023-08-11T15:53:15.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 738838 11.39 MB/sec execute 211 sec latency 16.301 ms 2023-08-11T15:53:15.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:15 smithi032 ceph-mon[97103]: daemon mds.h restarted 2023-08-11T15:53:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:16 smithi134 ceph-mon[130447]: pgmap v205: 129 pgs: 129 active+clean; 68 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-08-11T15:53:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:16 smithi134 ceph-mon[130447]: osdmap e99: 12 total, 12 up, 12 in 2023-08-11T15:53:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:16 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6836/805711177,v1:172.21.15.134:6837/805711177] up:boot 2023-08-11T15:53:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:16 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:53:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:16 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:53:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:16 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:53:16.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 740224 11.37 MB/sec execute 212 sec latency 134.866 ms 2023-08-11T15:53:16.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:16 smithi032 ceph-mon[97103]: pgmap v205: 129 pgs: 129 active+clean; 68 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-08-11T15:53:16.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:16 smithi032 ceph-mon[97103]: osdmap e99: 12 total, 12 up, 12 in 2023-08-11T15:53:16.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:16 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6836/805711177,v1:172.21.15.134:6837/805711177] up:boot 2023-08-11T15:53:16.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:16 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:53:16.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:16 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:53:16.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:16 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:53:16.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:16 smithi192 ceph-mon[130806]: pgmap v205: 129 pgs: 129 active+clean; 68 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-08-11T15:53:16.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:16 smithi192 ceph-mon[130806]: osdmap e99: 12 total, 12 up, 12 in 2023-08-11T15:53:16.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:16 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6836/805711177,v1:172.21.15.134:6837/805711177] up:boot 2023-08-11T15:53:16.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:16 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:53:16.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:16 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:53:16.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:16 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:53:17.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 742366 11.38 MB/sec execute 213 sec latency 306.537 ms 2023-08-11T15:53:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:17 smithi032 ceph-mon[97103]: osdmap e100: 12 total, 12 up, 12 in 2023-08-11T15:53:18.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:17 smithi032 ceph-mon[97103]: pgmap v208: 129 pgs: 129 active+clean; 64 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-08-11T15:53:18.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:17 smithi192 ceph-mon[130806]: osdmap e100: 12 total, 12 up, 12 in 2023-08-11T15:53:18.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:17 smithi192 ceph-mon[130806]: pgmap v208: 129 pgs: 129 active+clean; 64 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-08-11T15:53:18.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:17 smithi134 ceph-mon[130447]: osdmap e100: 12 total, 12 up, 12 in 2023-08-11T15:53:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:17 smithi134 ceph-mon[130447]: pgmap v208: 129 pgs: 129 active+clean; 64 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 424 op/s 2023-08-11T15:53:18.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 744024 11.35 MB/sec execute 214 sec latency 38.669 ms 2023-08-11T15:53:19.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 746287 11.36 MB/sec execute 215 sec latency 57.623 ms 2023-08-11T15:53:20.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:20 smithi192 ceph-mon[130806]: pgmap v209: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 487 op/s 2023-08-11T15:53:20.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:20 smithi134 ceph-mon[130447]: pgmap v209: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 487 op/s 2023-08-11T15:53:20.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 749011 11.37 MB/sec execute 216 sec latency 12.798 ms 2023-08-11T15:53:20.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:20 smithi032 ceph-mon[97103]: pgmap v209: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 487 op/s 2023-08-11T15:53:21.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 750828 11.37 MB/sec execute 217 sec latency 15.109 ms 2023-08-11T15:53:22.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:22 smithi192 ceph-mon[130806]: pgmap v210: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.1 MiB/s wr, 207 op/s 2023-08-11T15:53:22.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:22 smithi134 ceph-mon[130447]: pgmap v210: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.1 MiB/s wr, 207 op/s 2023-08-11T15:53:22.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 753134 11.38 MB/sec execute 218 sec latency 42.243 ms 2023-08-11T15:53:22.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:22 smithi032 ceph-mon[97103]: pgmap v210: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.1 MiB/s wr, 207 op/s 2023-08-11T15:53:23.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 755230 11.36 MB/sec execute 219 sec latency 26.479 ms 2023-08-11T15:53:24.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:24 smithi192 ceph-mon[130806]: pgmap v211: 129 pgs: 129 active+clean; 66 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2023-08-11T15:53:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:24 smithi134 ceph-mon[130447]: pgmap v211: 129 pgs: 129 active+clean; 66 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2023-08-11T15:53:24.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 757231 11.37 MB/sec execute 220 sec latency 12.100 ms 2023-08-11T15:53:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:24 smithi032 ceph-mon[97103]: pgmap v211: 129 pgs: 129 active+clean; 66 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2023-08-11T15:53:25.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 760003 11.39 MB/sec execute 221 sec latency 15.466 ms 2023-08-11T15:53:26.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:26 smithi134 ceph-mon[130447]: pgmap v212: 129 pgs: 129 active+clean; 66 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2023-08-11T15:53:26.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 761395 11.37 MB/sec execute 222 sec latency 43.296 ms 2023-08-11T15:53:26.785 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:26 smithi192 ceph-mon[130806]: pgmap v212: 129 pgs: 129 active+clean; 66 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2023-08-11T15:53:26.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:26 smithi032 ceph-mon[97103]: pgmap v212: 129 pgs: 129 active+clean; 66 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2023-08-11T15:53:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 -- 172.21.15.32:0/2326180116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581008b0 msgr2=0x7fc958100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 --2- 172.21.15.32:0/2326180116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581008b0 0x7fc958100d30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc948009990 tx=0x7fc948009ca0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 -- 172.21.15.32:0/2326180116 shutdown_connections 2023-08-11T15:53:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 --2- 172.21.15.32:0/2326180116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9581068c0 0x7fc958106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 --2- 172.21.15.32:0/2326180116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc958101270 0x7fc95810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 --2- 172.21.15.32:0/2326180116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581008b0 0x7fc958100d30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 -- 172.21.15.32:0/2326180116 >> 172.21.15.32:0/2326180116 conn(0x7fc9580fc380 msgr2=0x7fc9580fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 -- 172.21.15.32:0/2326180116 shutdown_connections 2023-08-11T15:53:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.548+0000 7fc9605ff700 1 -- 172.21.15.32:0/2326180116 wait complete. 2023-08-11T15:53:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.549+0000 7fc9605ff700 1 Processor -- start 2023-08-11T15:53:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.549+0000 7fc9605ff700 1 -- start start 2023-08-11T15:53:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.549+0000 7fc9605ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9581008b0 0x7fc95819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.549+0000 7fc9605ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc958101270 0x7fc95819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc9605ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581068c0 0x7fc958196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc9605ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958196e70 con 0x7fc958101270 2023-08-11T15:53:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc9605ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958196fe0 con 0x7fc9581008b0 2023-08-11T15:53:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc9605ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9581971a0 con 0x7fc9581068c0 2023-08-11T15:53:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9581008b0 0x7fc95819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9581008b0 0x7fc95819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50034/0 (socket says 172.21.15.32:50034) 2023-08-11T15:53:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 -- 172.21.15.32:0/3221714751 learned_addr learned my addr 172.21.15.32:0/3221714751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc958101270 0x7fc95819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581068c0 0x7fc958196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 -- 172.21.15.32:0/3221714751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581068c0 msgr2=0x7fc958196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581068c0 0x7fc958196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 -- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc958101270 msgr2=0x7fc95819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc958101270 0x7fc95819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 -- 172.21.15.32:0/3221714751 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480095f0 con 0x7fc9581008b0 2023-08-11T15:53:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.550+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9581008b0 0x7fc95819c9a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d980 tx=0x7fc95400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.551+0000 7fc94f7fe700 1 -- 172.21.15.32:0/3221714751 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954004220 con 0x7fc9581008b0 2023-08-11T15:53:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.551+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc958197480 con 0x7fc9581008b0 2023-08-11T15:53:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.551+0000 7fc94f7fe700 1 -- 172.21.15.32:0/3221714751 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc954007640 con 0x7fc9581008b0 2023-08-11T15:53:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.551+0000 7fc94f7fe700 1 -- 172.21.15.32:0/3221714751 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954005690 con 0x7fc9581008b0 2023-08-11T15:53:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.551+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9581a3990 con 0x7fc9581008b0 2023-08-11T15:53:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.554+0000 7fc94f7fe700 1 -- 172.21.15.32:0/3221714751 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fc954004750 con 0x7fc9581008b0 2023-08-11T15:53:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 Processor -- start 2023-08-11T15:53:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc94f7fe700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9440a00c0 0x7fc9440a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 -- start start 2023-08-11T15:53:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc94f7fe700 1 -- 172.21.15.32:0/3221714751 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fc9540cb750 con 0x7fc9581008b0 2023-08-11T15:53:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a43260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9440a00c0 0x7fc9440a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a3dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3dff0 0x564ad3a49c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad39965d0 con 0x564ad3a3dff0 2023-08-11T15:53:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3993ef0 con 0x564ad3a42e80 2023-08-11T15:53:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.556+0000 7fc973e31b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad398f7e0 con 0x564ad3a3d630 2023-08-11T15:53:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a43260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a43260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50040/0 (socket says 172.21.15.32:50040) 2023-08-11T15:53:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 -- 172.21.15.32:0/109051020 learned_addr learned my addr 172.21.15.32:0/109051020 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95db9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a3dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3dff0 0x564ad3a49c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 -- 172.21.15.32:0/109051020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 msgr2=0x564ad3a3dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a3dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 -- 172.21.15.32:0/109051020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3dff0 msgr2=0x564ad3a49c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3dff0 0x564ad3a49c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 -- 172.21.15.32:0/109051020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ad380f5d0 con 0x564ad3a42e80 2023-08-11T15:53:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95db9a700 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a3dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.557+0000 7fc95e39b700 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a43260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc954095500 tx=0x7fc954099210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaf10e88c421d015 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9440a00c0 0x7fc9440a2580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc95819d6a0 tx=0x7fc95000fc00 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc94cbfe700 1 -- 172.21.15.32:0/109051020 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95409c8c0 con 0x564ad3a42e80 2023-08-11T15:53:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc94cbfe700 1 -- 172.21.15.32:0/109051020 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9540a08a0 con 0x564ad3a42e80 2023-08-11T15:53:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc94cbfe700 1 -- 172.21.15.32:0/109051020 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954098f50 con 0x564ad3a42e80 2023-08-11T15:53:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc973e31b80 1 -- 172.21.15.32:0/109051020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 msgr2=0x564ad3a43260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc973e31b80 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a43260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc954095500 tx=0x7fc954099210 comp rx=0 tx=0).stop 2023-08-11T15:53:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc973e31b80 1 -- 172.21.15.32:0/109051020 shutdown_connections 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc973e31b80 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3dff0 0x564ad3a49c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc973e31b80 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a43260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc973e31b80 1 --2- 172.21.15.32:0/109051020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a3dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.558+0000 7fc973e31b80 1 -- 172.21.15.32:0/109051020 >> 172.21.15.32:0/109051020 conn(0x564ad396a8f0 msgr2=0x564ad3a37e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.559+0000 7fc973e31b80 1 -- 172.21.15.32:0/109051020 shutdown_connections 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.559+0000 7fc973e31b80 1 -- 172.21.15.32:0/109051020 wait complete. 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.559+0000 7fc973e31b80 1 -- start start 2023-08-11T15:53:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.559+0000 7fc973e31b80 1 Processor -- start 2023-08-11T15:53:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3d630 0x564ad3a50ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3a51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a4b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc973e31b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad392da60 con 0x564ad3a3d630 2023-08-11T15:53:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc973e31b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad392d760 con 0x564ad3a42e80 2023-08-11T15:53:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc973e31b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad38118c0 con 0x564ad3a3dff0 2023-08-11T15:53:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc973e31b80 10 client.0 _ll_register_callbacks cb 0x564ad3930a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3d630 0x564ad3a50ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc95db9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3a51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3d630 0x564ad3a50ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35032/0 (socket says 172.21.15.32:35032) 2023-08-11T15:53:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc95db9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3a51400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43628/0 (socket says 172.21.15.32:43628) 2023-08-11T15:53:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc95e39b700 1 -- 172.21.15.32:0/3369756350 learned_addr learned my addr 172.21.15.32:0/3369756350 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.560+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a4b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc95db9a700 1 -- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 msgr2=0x564ad3a4b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a4b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc95db9a700 1 -- 172.21.15.32:0/3369756350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3d630 msgr2=0x564ad3a50ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3d630 0x564ad3a50ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc95db9a700 1 -- 172.21.15.32:0/3369756350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95810e080 con 0x564ad3a3dff0 2023-08-11T15:53:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3d630 0x564ad3a50ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3a51400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc94800b2e0 tx=0x7fc94800b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35a0f928e00bcf0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc941ffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:53:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc941ffb700 1 -- 172.21.15.32:0/3369756350 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94801d070 con 0x564ad3a3dff0 2023-08-11T15:53:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ad3a307e0 con 0x564ad3a3dff0 2023-08-11T15:53:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc941ffb700 1 -- 172.21.15.32:0/3369756350 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fc948004700 con 0x564ad3a3dff0 2023-08-11T15:53:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc941ffb700 1 -- 172.21.15.32:0/3369756350 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9480056f0 con 0x564ad3a3dff0 2023-08-11T15:53:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.561+0000 7fc973e31b80 10 client.25951 fetch_fsmap learned FSMap version 31 2023-08-11T15:53:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fc9581a3990 con 0x564ad3a3dff0 2023-08-11T15:53:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc941ffb700 1 -- 172.21.15.32:0/3369756350 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ad3a307e0 con 0x564ad3a3dff0 2023-08-11T15:53:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc941ffb700 1 -- 172.21.15.32:0/3369756350 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc94800f8b0 con 0x564ad3a3dff0 2023-08-11T15:53:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc973e31b80 10 client.25951 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:53:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc973e31b80 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc973e31b80 20 client.25951 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3a5b3e0 0x564ad3a5d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc973e31b80 4 client.25951 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:53:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ad3a0c480 con 0x564ad3a5b3e0 2023-08-11T15:53:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.562+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3a5b3e0 0x564ad3a5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.563+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3a5b3e0 0x564ad3a5d800 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4cc0fa52f266db server_cookie=eeb51b722d690bf1 in_seq=0 out_seq=0 2023-08-11T15:53:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.563+0000 7fc941ffb700 10 client.25951 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc941ffb700 1 -- 172.21.15.32:0/3369756350 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x564ad3a0c480 con 0x564ad3a5b3e0 2023-08-11T15:53:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc941ffb700 10 client.25951 handle_command_reply: tid=0 2023-08-11T15:53:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 client.25951 shutdown 2023-08-11T15:53:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 msgr2=0x564ad3a51400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3a51400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc94800b2e0 tx=0x7fc94800b650 comp rx=0 tx=0).stop 2023-08-11T15:53:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 shutdown_connections 2023-08-11T15:53:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3a5b3e0 0x564ad3a5d800 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a3d630 0x564ad3a50ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a42e80 0x564ad3a4b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3369756350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3a51400 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc941ffb700 0 client.25951 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 >> 172.21.15.32:0/3369756350 conn(0x564ad396a8f0 msgr2=0x564ad3a375d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 shutdown_connections 2023-08-11T15:53:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.564+0000 7fc973e31b80 1 -- 172.21.15.32:0/3369756350 wait complete. 2023-08-11T15:53:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.565+0000 7fc973e31b80 20 client.25951 trim_cache size 0 max 16384 2023-08-11T15:53:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.580+0000 7fc973e31b80 1 Processor -- start 2023-08-11T15:53:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc973e31b80 1 -- start start 2023-08-11T15:53:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a48e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3dff0 0x564ad3a3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3a50a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc973e31b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3a38760 con 0x564ad3a42e80 2023-08-11T15:53:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc973e31b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3a43790 con 0x564ad3a3dff0 2023-08-11T15:53:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc973e31b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3a39680 con 0x564ad3a3d630 2023-08-11T15:53:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a48e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95db9a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3dff0 0x564ad3a3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a48e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43630/0 (socket says 172.21.15.32:43630) 2023-08-11T15:53:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95e39b700 1 -- 172.21.15.32:0/3743736354 learned_addr learned my addr 172.21.15.32:0/3743736354 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3a50a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95db9a700 1 -- 172.21.15.32:0/3743736354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 msgr2=0x564ad3a48e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a48e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.581+0000 7fc95db9a700 1 -- 172.21.15.32:0/3743736354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 msgr2=0x564ad3a50a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.582+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3a50a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.582+0000 7fc95db9a700 1 -- 172.21.15.32:0/3743736354 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ad37bbd80 con 0x564ad3a3dff0 2023-08-11T15:53:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.582+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3a50a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.582+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3dff0 0x564ad3a3da10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc94800fc10 tx=0x7fc9480244d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81cd3a2f0a1b2f46 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.582+0000 7fc94d3ff700 1 -- 172.21.15.32:0/3743736354 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94801d070 con 0x564ad3a3dff0 2023-08-11T15:53:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.582+0000 7fc94d3ff700 1 -- 172.21.15.32:0/3743736354 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc94801e040 con 0x564ad3a3dff0 2023-08-11T15:53:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.582+0000 7fc94d3ff700 1 -- 172.21.15.32:0/3743736354 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948024890 con 0x564ad3a3dff0 2023-08-11T15:53:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 -- 172.21.15.32:0/3743736354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3dff0 msgr2=0x564ad3a3da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3dff0 0x564ad3a3da10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc94800fc10 tx=0x7fc9480244d0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 -- 172.21.15.32:0/3743736354 shutdown_connections 2023-08-11T15:53:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3a50a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3d630 0x564ad3a48e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3743736354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3dff0 0x564ad3a3da10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 -- 172.21.15.32:0/3743736354 >> 172.21.15.32:0/3743736354 conn(0x564ad396a8f0 msgr2=0x564ad3a4aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 -- 172.21.15.32:0/3743736354 shutdown_connections 2023-08-11T15:53:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 -- 172.21.15.32:0/3743736354 wait complete. 2023-08-11T15:53:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.583+0000 7fc973e31b80 1 -- start start 2023-08-11T15:53:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 1 Processor -- start 2023-08-11T15:53:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3d630 0x564ad3aa3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3aa3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3aae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3aa3d50 con 0x564ad3a42e80 2023-08-11T15:53:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3aa3ec0 con 0x564ad3a3d630 2023-08-11T15:53:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3aae630 con 0x564ad3a3dff0 2023-08-11T15:53:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc973e31b80 10 client.0 _ll_register_callbacks cb 0x564ad3930a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3d630 0x564ad3aa3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc95e39b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3d630 0x564ad3aa3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:50062/0 (socket says 172.21.15.32:50062) 2023-08-11T15:53:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc95e39b700 1 -- 172.21.15.32:0/3052381189 learned_addr learned my addr 172.21.15.32:0/3052381189 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc95eb9c700 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3aae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.584+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3aa3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc95e39b700 1 -- 172.21.15.32:0/3052381189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 msgr2=0x564ad3aa3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3aa3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc95e39b700 1 -- 172.21.15.32:0/3052381189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 msgr2=0x564ad3aae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3aae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc95e39b700 1 -- 172.21.15.32:0/3052381189 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9581a3990 con 0x564ad3a3d630 2023-08-11T15:53:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc95e39b700 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3d630 0x564ad3aa3040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc954094f90 tx=0x7fc954099300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d55ca7e6198c5f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc941ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc941ffb700 1 -- 172.21.15.32:0/3052381189 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540b6040 con 0x564ad3a3d630 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9540a08a0 con 0x564ad3a3d630 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc941ffb700 1 -- 172.21.15.32:0/3052381189 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fc9540cb300 con 0x564ad3a3d630 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc941ffb700 1 -- 172.21.15.32:0/3052381189 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc954098860 con 0x564ad3a3d630 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc973e31b80 10 client.25955 fetch_fsmap learned FSMap version 31 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.585+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564ad37a4c00 con 0x564ad3a3d630 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc941ffb700 1 -- 172.21.15.32:0/3052381189 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9540a08a0 con 0x564ad3a3d630 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc941ffb700 1 -- 172.21.15.32:0/3052381189 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc9540c8d50 con 0x564ad3a3d630 2023-08-11T15:53:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc973e31b80 10 client.25955 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:53:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc973e31b80 10 client.25955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc973e31b80 20 client.25955 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3aaf980 0x564ad3ab1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc973e31b80 4 client.25955 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:53:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ad39887c0 con 0x564ad3aaf980 2023-08-11T15:53:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.586+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3aaf980 0x564ad3ab1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.587+0000 7fc95db9a700 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3aaf980 0x564ad3ab1d70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=563df123a04e3dd2 server_cookie=6cc3702872e4dd04 in_seq=0 out_seq=0 2023-08-11T15:53:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.587+0000 7fc941ffb700 10 client.25955 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:27.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 763644 11.38 MB/sec execute 223 sec latency 140.202 ms 2023-08-11T15:53:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc941ffb700 1 -- 172.21.15.32:0/3052381189 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ad39887c0 con 0x564ad3aaf980 2023-08-11T15:53:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc941ffb700 10 client.25955 handle_command_reply: tid=0 2023-08-11T15:53:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 client.25955 shutdown 2023-08-11T15:53:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3d630 msgr2=0x564ad3aa3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3d630 0x564ad3aa3040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc954094f90 tx=0x7fc954099300 comp rx=0 tx=0).stop 2023-08-11T15:53:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 shutdown_connections 2023-08-11T15:53:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x564ad3aaf980 0x564ad3ab1d70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ad3a42e80 0x564ad3aae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ad3a3d630 0x564ad3aa3040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc941ffb700 0 client.25955 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 --2- 172.21.15.32:0/3052381189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad3a3dff0 0x564ad3aa3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 >> 172.21.15.32:0/3052381189 conn(0x564ad396a8f0 msgr2=0x564ad3a499c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 shutdown_connections 2023-08-11T15:53:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.923+0000 7fc973e31b80 1 -- 172.21.15.32:0/3052381189 wait complete. 2023-08-11T15:53:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc973e31b80 20 client.25955 trim_cache size 0 max 16384 2023-08-11T15:53:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:53:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:53:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:53:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:53:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9440a00c0 msgr2=0x7fc9440a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9440a00c0 0x7fc9440a2580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc95819d6a0 tx=0x7fc95000fc00 comp rx=0 tx=0).stop 2023-08-11T15:53:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9581008b0 msgr2=0x7fc95819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9581008b0 0x7fc95819c9a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d980 tx=0x7fc95400dc90 comp rx=0 tx=0).stop 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 shutdown_connections 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9440a00c0 0x7fc9440a2580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9581068c0 0x7fc958196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9581008b0 0x7fc95819c9a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 --2- 172.21.15.32:0/3221714751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc958101270 0x7fc95819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 >> 172.21.15.32:0/3221714751 conn(0x7fc9580fc380 msgr2=0x7fc9580fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 shutdown_connections 2023-08-11T15:53:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:27.924+0000 7fc9605ff700 1 -- 172.21.15.32:0/3221714751 wait complete. 2023-08-11T15:53:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:27 smithi032 ceph-mon[97103]: pgmap v213: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 317 op/s 2023-08-11T15:53:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:27 smithi192 ceph-mon[130806]: pgmap v213: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 317 op/s 2023-08-11T15:53:28.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:27 smithi134 ceph-mon[130447]: pgmap v213: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 317 op/s 2023-08-11T15:53:28.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 764968 11.35 MB/sec execute 224 sec latency 267.443 ms 2023-08-11T15:53:29.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 767341 11.37 MB/sec execute 225 sec latency 28.410 ms 2023-08-11T15:53:30.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:30 smithi192 ceph-mon[130806]: pgmap v214: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-08-11T15:53:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:30 smithi134 ceph-mon[130447]: pgmap v214: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-08-11T15:53:30.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 768652 11.34 MB/sec execute 226 sec latency 171.577 ms 2023-08-11T15:53:30.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:30 smithi032 ceph-mon[97103]: pgmap v214: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-08-11T15:53:31.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 771084 11.35 MB/sec execute 227 sec latency 15.146 ms 2023-08-11T15:53:32.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:32 smithi192 ceph-mon[130806]: pgmap v215: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2023-08-11T15:53:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:32 smithi134 ceph-mon[130447]: pgmap v215: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2023-08-11T15:53:32.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 773509 11.35 MB/sec execute 228 sec latency 13.826 ms 2023-08-11T15:53:32.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:32 smithi032 ceph-mon[97103]: pgmap v215: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2023-08-11T15:53:33.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 775554 11.36 MB/sec execute 229 sec latency 26.144 ms 2023-08-11T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:34 smithi192 ceph-mon[130806]: pgmap v216: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 344 op/s 2023-08-11T15:53:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:34 smithi134 ceph-mon[130447]: pgmap v216: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 344 op/s 2023-08-11T15:53:34.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 778132 11.37 MB/sec execute 230 sec latency 11.269 ms 2023-08-11T15:53:34.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:34 smithi032 ceph-mon[97103]: pgmap v216: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 344 op/s 2023-08-11T15:53:35.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 780186 11.35 MB/sec execute 231 sec latency 91.416 ms 2023-08-11T15:53:36.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:36 smithi192 ceph-mon[130806]: pgmap v217: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2023-08-11T15:53:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:36 smithi134 ceph-mon[130447]: pgmap v217: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2023-08-11T15:53:36.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 782128 11.36 MB/sec execute 232 sec latency 20.495 ms 2023-08-11T15:53:36.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:36 smithi032 ceph-mon[97103]: pgmap v217: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2023-08-11T15:53:37.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 784916 11.38 MB/sec execute 233 sec latency 16.612 ms 2023-08-11T15:53:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:37 smithi032 ceph-mon[97103]: pgmap v218: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 339 op/s 2023-08-11T15:53:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:37 smithi192 ceph-mon[130806]: pgmap v218: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 339 op/s 2023-08-11T15:53:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:37 smithi134 ceph-mon[130447]: pgmap v218: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 339 op/s 2023-08-11T15:53:38.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 786376 11.36 MB/sec execute 234 sec latency 53.323 ms 2023-08-11T15:53:39.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 788775 11.37 MB/sec execute 235 sec latency 19.157 ms 2023-08-11T15:53:40.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:40 smithi192 ceph-mon[130806]: pgmap v219: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2023-08-11T15:53:40.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:40 smithi134 ceph-mon[130447]: pgmap v219: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2023-08-11T15:53:40.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 790836 11.36 MB/sec execute 236 sec latency 30.464 ms 2023-08-11T15:53:40.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:40 smithi032 ceph-mon[97103]: pgmap v219: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2023-08-11T15:53:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1757009455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 msgr2=0x7fc9e4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1757009455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e4100d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009990 tx=0x7fc9d8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1757009455 shutdown_connections 2023-08-11T15:53:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1757009455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e4101260 0x7fc9e410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1757009455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e41068b0 0x7fc9e4106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1757009455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e4100d20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1757009455 >> 172.21.15.32:0/1757009455 conn(0x7fc9e40fc370 msgr2=0x7fc9e40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1757009455 shutdown_connections 2023-08-11T15:53:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.080+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1757009455 wait complete. 2023-08-11T15:53:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.081+0000 7fc9e93ff700 1 Processor -- start 2023-08-11T15:53:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.081+0000 7fc9e93ff700 1 -- start start 2023-08-11T15:53:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e93ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e93ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e4101260 0x7fc9e419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e93ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e41068b0 0x7fc9e41967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e93ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4196e50 con 0x7fc9e4101260 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e93ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4196fc0 con 0x7fc9e41008a0 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e93ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4197180 con 0x7fc9e41068b0 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:54616/0 (socket says 172.21.15.32:54616) 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1694239006 learned_addr learned my addr 172.21.15.32:0/1694239006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e4101260 0x7fc9e419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e41068b0 0x7fc9e41967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1694239006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e41068b0 msgr2=0x7fc9e41967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e41068b0 0x7fc9e41967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e4101260 msgr2=0x7fc9e419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e4101260 0x7fc9e419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.082+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1694239006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80095f0 con 0x7fc9e41008a0 2023-08-11T15:53:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.083+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e419c980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e9a0 tx=0x7fc9d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.083+0000 7fc9cbfff700 1 -- 172.21.15.32:0/1694239006 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4009790 con 0x7fc9e41008a0 2023-08-11T15:53:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.083+0000 7fc9cbfff700 1 -- 172.21.15.32:0/1694239006 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9d4010430 con 0x7fc9e41008a0 2023-08-11T15:53:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.083+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e4197460 con 0x7fc9e41008a0 2023-08-11T15:53:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.083+0000 7fc9cbfff700 1 -- 172.21.15.32:0/1694239006 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4018680 con 0x7fc9e41008a0 2023-08-11T15:53:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.083+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e41a3970 con 0x7fc9e41008a0 2023-08-11T15:53:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.086+0000 7fc9cbfff700 1 -- 172.21.15.32:0/1694239006 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fc9d40105a0 con 0x7fc9e41008a0 2023-08-11T15:53:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.087+0000 7fc9cbfff700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9cc0a0140 0x7fc9cc0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.088+0000 7fc9cbfff700 1 -- 172.21.15.32:0/1694239006 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fc9d4014070 con 0x7fc9e41008a0 2023-08-11T15:53:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.088+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9cc0a0140 0x7fc9cc0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.088+0000 7fc9fcdd1b80 1 Processor -- start 2023-08-11T15:53:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.088+0000 7fc9fcdd1b80 1 -- start start 2023-08-11T15:53:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d2941ac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9cc0a0140 0x7fc9cc0a2600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4105fd0 tx=0x7fc9d8005fb0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2940ff5d0 con 0x55d2941abe80 2023-08-11T15:53:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2940fcef0 con 0x55d2941a6630 2023-08-11T15:53:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2940f87e0 con 0x55d2941a6ff0 2023-08-11T15:53:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e27fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e27fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:54624/0 (socket says 172.21.15.32:54624) 2023-08-11T15:53:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e27fc700 1 -- 172.21.15.32:0/546431259 learned_addr learned my addr 172.21.15.32:0/546431259 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d2941ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e37fe700 1 -- 172.21.15.32:0/546431259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 msgr2=0x55d2941a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e37fe700 1 -- 172.21.15.32:0/546431259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 msgr2=0x55d2941ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d2941ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e37fe700 1 -- 172.21.15.32:0/546431259 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d293f785d0 con 0x55d2941a6ff0 2023-08-11T15:53:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.089+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d2941ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.090+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b2c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc011550 tx=0x7fc9dc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9521d453eb70c0cc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.090+0000 7fc9c97fa700 1 -- 172.21.15.32:0/546431259 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00e040 con 0x55d2941a6ff0 2023-08-11T15:53:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.090+0000 7fc9c97fa700 1 -- 172.21.15.32:0/546431259 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9dc00f040 con 0x55d2941a6ff0 2023-08-11T15:53:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.090+0000 7fc9c97fa700 1 -- 172.21.15.32:0/546431259 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc004410 con 0x55d2941a6ff0 2023-08-11T15:53:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/546431259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 msgr2=0x55d2941b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b2c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc011550 tx=0x7fc9dc009920 comp rx=0 tx=0).stop 2023-08-11T15:53:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/546431259 shutdown_connections 2023-08-11T15:53:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d2941ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b2c10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/546431259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/546431259 >> 172.21.15.32:0/546431259 conn(0x55d2940d38f0 msgr2=0x55d2941a0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/546431259 shutdown_connections 2023-08-11T15:53:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/546431259 wait complete. 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.091+0000 7fc9fcdd1b80 1 -- start start 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 1 Processor -- start 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6630 0x55d2941b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941abe80 0x55d2941afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d294096a60 con 0x55d2941a6630 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d294096760 con 0x55d2941abe80 2023-08-11T15:53:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d293f7a8c0 con 0x55d2941a6ff0 2023-08-11T15:53:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9fcdd1b80 10 client.0 _ll_register_callbacks cb 0x55d294099a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6630 0x55d2941b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9e27fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9e37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941abe80 0x55d2941afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9e37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941abe80 0x55d2941afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:54636/0 (socket says 172.21.15.32:54636) 2023-08-11T15:53:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9e27fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:55828/0 (socket says 172.21.15.32:55828) 2023-08-11T15:53:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6630 0x55d2941b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52952/0 (socket says 172.21.15.32:52952) 2023-08-11T15:53:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.092+0000 7fc9e37fe700 1 -- 172.21.15.32:0/3680109985 learned_addr learned my addr 172.21.15.32:0/3680109985 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9e37fe700 1 -- 172.21.15.32:0/3680109985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 msgr2=0x55d2941b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9e37fe700 1 -- 172.21.15.32:0/3680109985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6630 msgr2=0x55d2941b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6630 0x55d2941b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9e37fe700 1 -- 172.21.15.32:0/3680109985 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e41a3970 con 0x55d2941abe80 2023-08-11T15:53:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941abe80 0x55d2941afd30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc011f10 tx=0x7fc9dc011f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf4ca462c475beb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/3680109985 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00e040 con 0x55d2941abe80 2023-08-11T15:53:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2941997e0 con 0x55d2941abe80 2023-08-11T15:53:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.093+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/3680109985 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fc9dc004ad0 con 0x55d2941abe80 2023-08-11T15:53:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/3680109985 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9dc0040d0 con 0x55d2941abe80 2023-08-11T15:53:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 10 client.25967 fetch_fsmap learned FSMap version 31 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fc9dc011230 con 0x55d2941abe80 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/3680109985 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2941997e0 con 0x55d2941abe80 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/3680109985 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc9dc01b020 con 0x55d2941abe80 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 20 client.25967 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d2941bfee0 0x55d2941c2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 4 client.25967 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:53:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.094+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d294175480 con 0x55d2941bfee0 2023-08-11T15:53:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.095+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d2941bfee0 0x55d2941c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.095+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d2941bfee0 0x55d2941c2300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b9a07c6ec968a2 server_cookie=e1f29ed6776d25c2 in_seq=0 out_seq=0 2023-08-11T15:53:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.095+0000 7fc9c6ff5700 10 client.25967 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/3680109985 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d294175480 con 0x55d2941bfee0 2023-08-11T15:53:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9c6ff5700 10 client.25967 handle_command_reply: tid=0 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 client.25967 shutdown 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941abe80 msgr2=0x55d2941afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941abe80 0x55d2941afd30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc011f10 tx=0x7fc9dc011f40 comp rx=0 tx=0).stop 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 shutdown_connections 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d2941bfee0 0x55d2941c2300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6630 0x55d2941b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941abe80 0x55d2941afd30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/3680109985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6ff0 0x55d2941b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 >> 172.21.15.32:0/3680109985 conn(0x55d2940d38f0 msgr2=0x55d2941a05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9c6ff5700 0 client.25967 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.096+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 shutdown_connections 2023-08-11T15:53:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.097+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/3680109985 wait complete. 2023-08-11T15:53:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.097+0000 7fc9fcdd1b80 20 client.25967 trim_cache size 0 max 16384 2023-08-11T15:53:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.112+0000 7fc9fcdd1b80 1 Processor -- start 2023-08-11T15:53:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.112+0000 7fc9fcdd1b80 1 -- start start 2023-08-11T15:53:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6ff0 0x55d2941b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941abe80 0x55d2941a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2941abb00 con 0x55d2941a6ff0 2023-08-11T15:53:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2941ae2d0 con 0x55d2941a6630 2023-08-11T15:53:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2941aaae0 con 0x55d2941abe80 2023-08-11T15:53:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:54650/0 (socket says 172.21.15.32:54650) 2023-08-11T15:53:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1898893779 learned_addr learned my addr 172.21.15.32:0/1898893779 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941abe80 0x55d2941a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6ff0 0x55d2941b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1898893779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941abe80 msgr2=0x55d2941a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941abe80 0x55d2941a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1898893779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6ff0 msgr2=0x55d2941b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6ff0 0x55d2941b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e2ffd700 1 -- 172.21.15.32:0/1898893779 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d293f24d80 con 0x55d2941a6630 2023-08-11T15:53:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e37fe700 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941abe80 0x55d2941a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.113+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6ff0 0x55d2941b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.114+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941c6c40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e970 tx=0x7fc9d40cb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fd7575705fcc11 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.114+0000 7fc9c9ffb700 1 -- 172.21.15.32:0/1898893779 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40b0070 con 0x55d2941a6630 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.114+0000 7fc9c9ffb700 1 -- 172.21.15.32:0/1898893779 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9d409cc10 con 0x55d2941a6630 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.114+0000 7fc9c9ffb700 1 -- 172.21.15.32:0/1898893779 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4099040 con 0x55d2941a6630 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/1898893779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 msgr2=0x55d2941c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941c6c40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e970 tx=0x7fc9d40cb040 comp rx=0 tx=0).stop 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/1898893779 shutdown_connections 2023-08-11T15:53:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941abe80 0x55d2941a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6630 0x55d2941c6c40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/1898893779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941a6ff0 0x55d2941b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/1898893779 >> 172.21.15.32:0/1898893779 conn(0x55d2940d38f0 msgr2=0x55d2941a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/1898893779 shutdown_connections 2023-08-11T15:53:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/1898893779 wait complete. 2023-08-11T15:53:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.115+0000 7fc9fcdd1b80 1 -- start start 2023-08-11T15:53:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 1 Processor -- start 2023-08-11T15:53:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6630 0x55d294207b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6ff0 0x55d294208040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d294208580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d294212c60 con 0x55d2941abe80 2023-08-11T15:53:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d294212da0 con 0x55d2941a6ff0 2023-08-11T15:53:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d294212f60 con 0x55d2941a6630 2023-08-11T15:53:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9fcdd1b80 10 client.0 _ll_register_callbacks cb 0x7fc9d4095160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9e27fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6ff0 0x55d294208040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6630 0x55d294207b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9e27fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6ff0 0x55d294208040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:54658/0 (socket says 172.21.15.32:54658) 2023-08-11T15:53:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6630 0x55d294207b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:55846/0 (socket says 172.21.15.32:55846) 2023-08-11T15:53:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9e27fc700 1 -- 172.21.15.32:0/347478505 learned_addr learned my addr 172.21.15.32:0/347478505 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.116+0000 7fc9e37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d294208580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9e27fc700 1 -- 172.21.15.32:0/347478505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6630 msgr2=0x55d294207b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6630 0x55d294207b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9e27fc700 1 -- 172.21.15.32:0/347478505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 msgr2=0x55d294208580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d294208580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9e27fc700 1 -- 172.21.15.32:0/347478505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc011230 con 0x55d2941a6ff0 2023-08-11T15:53:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6630 0x55d294207b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9e27fc700 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6ff0 0x55d294208040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005f50 tx=0x7fc9d801a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94227b86776b6a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/347478505 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d802d6d0 con 0x55d2941a6ff0 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9dc00f040 con 0x55d2941a6ff0 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.117+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/347478505 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fc9d802d830 con 0x55d2941a6ff0 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/347478505 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9d8004e50 con 0x55d2941a6ff0 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 10 client.25973 fetch_fsmap learned FSMap version 31 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fc9e4197460 con 0x55d2941a6ff0 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/347478505 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9dc00f040 con 0x55d2941a6ff0 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/347478505 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc9d800fb70 con 0x55d2941a6ff0 2023-08-11T15:53:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:53:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 20 client.25973 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d294214770 0x55d294216b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 4 client.25973 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:53:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.118+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d2940f17c0 con 0x55d294214770 2023-08-11T15:53:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.119+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d294214770 0x55d294216b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.119+0000 7fc9e2ffd700 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d294214770 0x55d294216b90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfbda49ab837566 server_cookie=5306d32efe0b7459 in_seq=0 out_seq=0 2023-08-11T15:53:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.119+0000 7fc9c6ff5700 10 client.25973 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9c6ff5700 1 -- 172.21.15.32:0/347478505 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d2940f17c0 con 0x55d294214770 2023-08-11T15:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9c6ff5700 10 client.25973 handle_command_reply: tid=0 2023-08-11T15:53:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 client.25973 shutdown 2023-08-11T15:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6ff0 msgr2=0x55d294208040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6ff0 0x55d294208040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005f50 tx=0x7fc9d801a050 comp rx=0 tx=0).stop 2023-08-11T15:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 shutdown_connections 2023-08-11T15:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55d294214770 0x55d294216b90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d2941abe80 0x55d294208580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d2941a6630 0x55d294207b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 --2- 172.21.15.32:0/347478505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2941a6ff0 0x55d294208040 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005f50 tx=0x7fc9d801a050 comp rx=0 tx=0).stop 2023-08-11T15:53:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 >> 172.21.15.32:0/347478505 conn(0x55d2940d38f0 msgr2=0x55d2941aaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.437+0000 7fc9c6ff5700 0 client.25973 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.438+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 shutdown_connections 2023-08-11T15:53:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.438+0000 7fc9fcdd1b80 1 -- 172.21.15.32:0/347478505 wait complete. 2023-08-11T15:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.438+0000 7fc9fcdd1b80 20 client.25973 trim_cache size 0 max 16384 2023-08-11T15:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:53:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9cc0a0140 msgr2=0x7fc9cc0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9cc0a0140 0x7fc9cc0a2600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4105fd0 tx=0x7fc9d8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 msgr2=0x7fc9e419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e419c980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e9a0 tx=0x7fc9d400ecb0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 shutdown_connections 2023-08-11T15:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc9cc0a0140 0x7fc9cc0a2600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e41068b0 0x7fc9e41967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9e41008a0 0x7fc9e419c980 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 --2- 172.21.15.32:0/1694239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e4101260 0x7fc9e419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.439+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 >> 172.21.15.32:0/1694239006 conn(0x7fc9e40fc370 msgr2=0x7fc9e40fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.440+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 shutdown_connections 2023-08-11T15:53:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:41.440+0000 7fc9e93ff700 1 -- 172.21.15.32:0/1694239006 wait complete. 2023-08-11T15:53:41.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 792259 11.35 MB/sec execute 237 sec latency 137.336 ms 2023-08-11T15:53:42.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:42 smithi134 ceph-mon[130447]: pgmap v220: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2023-08-11T15:53:42.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 794545 11.35 MB/sec execute 238 sec latency 33.460 ms 2023-08-11T15:53:42.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:42 smithi032 ceph-mon[97103]: pgmap v220: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2023-08-11T15:53:42.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:42 smithi192 ceph-mon[130806]: pgmap v220: 129 pgs: 129 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2023-08-11T15:53:43.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 796475 11.35 MB/sec execute 239 sec latency 62.466 ms 2023-08-11T15:53:44.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:44 smithi134 ceph-mon[130447]: pgmap v221: 129 pgs: 129 active+clean; 65 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 470 op/s 2023-08-11T15:53:44.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 798330 11.34 MB/sec execute 240 sec latency 81.270 ms 2023-08-11T15:53:44.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:44 smithi032 ceph-mon[97103]: pgmap v221: 129 pgs: 129 active+clean; 65 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 470 op/s 2023-08-11T15:53:44.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:44 smithi192 ceph-mon[130806]: pgmap v221: 129 pgs: 129 active+clean; 65 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 470 op/s 2023-08-11T15:53:45.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 800297 11.35 MB/sec execute 241 sec latency 25.624 ms 2023-08-11T15:53:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:46 smithi134 ceph-mon[130447]: pgmap v222: 129 pgs: 129 active+clean; 65 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 265 op/s 2023-08-11T15:53:46.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 802735 11.36 MB/sec execute 242 sec latency 27.067 ms 2023-08-11T15:53:46.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:46 smithi032 ceph-mon[97103]: pgmap v222: 129 pgs: 129 active+clean; 65 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 265 op/s 2023-08-11T15:53:46.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:46 smithi192 ceph-mon[130806]: pgmap v222: 129 pgs: 129 active+clean; 65 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 265 op/s 2023-08-11T15:53:47.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 804846 11.34 MB/sec execute 243 sec latency 15.171 ms 2023-08-11T15:53:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:47 smithi032 ceph-mon[97103]: pgmap v223: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 358 op/s 2023-08-11T15:53:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:47 smithi192 ceph-mon[130806]: pgmap v223: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 358 op/s 2023-08-11T15:53:48.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:47 smithi134 ceph-mon[130447]: pgmap v223: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 358 op/s 2023-08-11T15:53:48.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 806760 11.34 MB/sec execute 244 sec latency 30.234 ms 2023-08-11T15:53:49.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 808830 11.34 MB/sec execute 245 sec latency 58.672 ms 2023-08-11T15:53:50.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:50 smithi192 ceph-mon[130806]: pgmap v224: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 306 op/s 2023-08-11T15:53:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:50 smithi134 ceph-mon[130447]: pgmap v224: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 306 op/s 2023-08-11T15:53:50.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 810480 11.33 MB/sec execute 246 sec latency 24.143 ms 2023-08-11T15:53:50.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:50 smithi032 ceph-mon[97103]: pgmap v224: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 306 op/s 2023-08-11T15:53:51.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 813204 11.35 MB/sec execute 247 sec latency 15.438 ms 2023-08-11T15:53:52.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:52 smithi192 ceph-mon[130806]: pgmap v225: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 302 op/s 2023-08-11T15:53:52.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:52 smithi134 ceph-mon[130447]: pgmap v225: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 302 op/s 2023-08-11T15:53:52.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 814233 11.32 MB/sec execute 248 sec latency 167.710 ms 2023-08-11T15:53:52.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:52 smithi032 ceph-mon[97103]: pgmap v225: 129 pgs: 129 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 302 op/s 2023-08-11T15:53:53.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 816885 11.34 MB/sec execute 249 sec latency 11.857 ms 2023-08-11T15:53:54.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:54 smithi192 ceph-mon[130806]: pgmap v226: 129 pgs: 129 active+clean; 79 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2023-08-11T15:53:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.631+0000 7f603efff700 1 -- 172.21.15.32:0/3426540826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 msgr2=0x7f6038107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.631+0000 7f603efff700 1 --2- 172.21.15.32:0/3426540826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 0x7f6038107920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6028009990 tx=0x7f6028009ca0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.636+0000 7f603efff700 1 -- 172.21.15.32:0/3426540826 shutdown_connections 2023-08-11T15:53:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.636+0000 7f603efff700 1 --2- 172.21.15.32:0/3426540826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 0x7f6038114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.636+0000 7f603efff700 1 --2- 172.21.15.32:0/3426540826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 0x7f6038107920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.636+0000 7f603efff700 1 --2- 172.21.15.32:0/3426540826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6038107e60 0x7f60381082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.636+0000 7f603efff700 1 -- 172.21.15.32:0/3426540826 >> 172.21.15.32:0/3426540826 conn(0x7f603806ce20 msgr2=0x7f603806d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.652+0000 7f603efff700 1 -- 172.21.15.32:0/3426540826 shutdown_connections 2023-08-11T15:53:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.654+0000 7f603efff700 1 -- 172.21.15.32:0/3426540826 wait complete. 2023-08-11T15:53:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.655+0000 7f603efff700 1 Processor -- start 2023-08-11T15:53:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.656+0000 7f603efff700 1 -- start start 2023-08-11T15:53:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.656+0000 7f603efff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 0x7f6038117210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.656+0000 7f603efff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6038107e60 0x7f6038117750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.656+0000 7f603efff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 0x7f603811cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.656+0000 7f603efff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603811d500 con 0x7f6038107e60 2023-08-11T15:53:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.656+0000 7f603efff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603811d670 con 0x7f60381121a0 2023-08-11T15:53:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.656+0000 7f603efff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603811d830 con 0x7f6038068160 2023-08-11T15:53:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.657+0000 7f603d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 0x7f603811cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.657+0000 7f603d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 0x7f603811cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:49004/0 (socket says 172.21.15.32:49004) 2023-08-11T15:53:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.657+0000 7f603d59c700 1 -- 172.21.15.32:0/3291033177 learned_addr learned my addr 172.21.15.32:0/3291033177 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.658+0000 7f603cd9b700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 0x7f6038117210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.658+0000 7f603d59c700 1 -- 172.21.15.32:0/3291033177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 msgr2=0x7f6038117210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.658+0000 7f603d59c700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 0x7f6038117210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.658+0000 7f603d59c700 1 -- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6038107e60 msgr2=0x7f6038117750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.658+0000 7f603d59c700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6038107e60 0x7f6038117750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.658+0000 7f603d59c700 1 -- 172.21.15.32:0/3291033177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280095f0 con 0x7f60381121a0 2023-08-11T15:53:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.659+0000 7f603d59c700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 0x7f603811cd30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6030007ee0 tx=0x7f603000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.660+0000 7f602dffb700 1 -- 172.21.15.32:0/3291033177 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030017070 con 0x7f60381121a0 2023-08-11T15:53:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.660+0000 7f602dffb700 1 -- 172.21.15.32:0/3291033177 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603000f040 con 0x7f60381121a0 2023-08-11T15:53:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.660+0000 7f602dffb700 1 -- 172.21.15.32:0/3291033177 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030005740 con 0x7f60381121a0 2023-08-11T15:53:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.660+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603811db10 con 0x7f60381121a0 2023-08-11T15:53:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.660+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60381bafb0 con 0x7f60381121a0 2023-08-11T15:53:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.669+0000 7f6052973b80 1 Processor -- start 2023-08-11T15:53:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.672+0000 7f6052973b80 1 -- start start 2023-08-11T15:53:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.673+0000 7f6052973b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 0x55e39387b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.673+0000 7f6052973b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e393875ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.673+0000 7f6052973b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875ff0 0x55e393881c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.673+0000 7f6052973b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3937ce5d0 con 0x55e39387ae80 2023-08-11T15:53:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.673+0000 7f6052973b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3937cbef0 con 0x55e393875ff0 2023-08-11T15:53:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.673+0000 7f6052973b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3937c77e0 con 0x55e393875630 2023-08-11T15:53:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.674+0000 7f603cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e393875ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.674+0000 7f603cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e393875ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43936/0 (socket says 172.21.15.32:43936) 2023-08-11T15:53:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.674+0000 7f603cd9b700 1 -- 172.21.15.32:0/390516461 learned_addr learned my addr 172.21.15.32:0/390516461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.675+0000 7f602dffb700 1 -- 172.21.15.32:0/3291033177 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f60300058a0 con 0x7f60381121a0 2023-08-11T15:53:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.676+0000 7f603cd9b700 1 -- 172.21.15.32:0/390516461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875ff0 msgr2=0x55e393881c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T15:53:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.676+0000 7f603cd9b700 1 --2- 172.21.15.32:0/390516461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875ff0 0x55e393881c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.676+0000 7f603cd9b700 1 -- 172.21.15.32:0/390516461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 msgr2=0x55e39387b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.676+0000 7f603cd9b700 1 --2- 172.21.15.32:0/390516461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 0x55e39387b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.676+0000 7f603cd9b700 1 -- 172.21.15.32:0/390516461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3936475d0 con 0x55e393875630 2023-08-11T15:53:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.678+0000 7f602dffb700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f60200a02d0 0x7f60200a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.678+0000 7f602dffb700 1 -- 172.21.15.32:0/3291033177 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f60300cb030 con 0x7f60381121a0 2023-08-11T15:53:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.681+0000 7f602ffff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f60200a02d0 0x7f60200a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.684+0000 7f602ffff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f60200a02d0 0x7f60200a2790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6034005fd0 tx=0x7f6034009510 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.688+0000 7f603cd9b700 1 --2- 172.21.15.32:0/390516461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e393875ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60280036c0 tx=0x7f60280036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f283aa497a5d17f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.691+0000 7f601effd700 1 -- 172.21.15.32:0/390516461 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028009ee0 con 0x55e393875630 2023-08-11T15:53:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.691+0000 7f601effd700 1 -- 172.21.15.32:0/390516461 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f602801e040 con 0x55e393875630 2023-08-11T15:53:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.691+0000 7f601effd700 1 -- 172.21.15.32:0/390516461 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280054c0 con 0x55e393875630 2023-08-11T15:53:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.692+0000 7f6052973b80 1 -- 172.21.15.32:0/390516461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 msgr2=0x55e393875ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.692+0000 7f6052973b80 1 --2- 172.21.15.32:0/390516461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e393875ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60280036c0 tx=0x7f60280036f0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.693+0000 7f6052973b80 1 -- 172.21.15.32:0/390516461 shutdown_connections 2023-08-11T15:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.693+0000 7f6052973b80 1 --2- 172.21.15.32:0/390516461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 0x55e39387b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.693+0000 7f6052973b80 1 --2- 172.21.15.32:0/390516461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875ff0 0x55e393881c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.693+0000 7f6052973b80 1 --2- 172.21.15.32:0/390516461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e393875ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.694+0000 7f6052973b80 1 -- 172.21.15.32:0/390516461 >> 172.21.15.32:0/390516461 conn(0x55e3937a28f0 msgr2=0x55e39386fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:54.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.699+0000 7f6052973b80 1 -- 172.21.15.32:0/390516461 shutdown_connections 2023-08-11T15:53:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.701+0000 7f6052973b80 1 -- 172.21.15.32:0/390516461 wait complete. 2023-08-11T15:53:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.702+0000 7f6052973b80 1 -- start start 2023-08-11T15:53:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 1 Processor -- start 2023-08-11T15:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875630 0x55e393884ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875ff0 0x55e393884ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 0x55e39387eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e393765a60 con 0x55e39387ae80 2023-08-11T15:53:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e393765760 con 0x55e393875630 2023-08-11T15:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3936498c0 con 0x55e393875ff0 2023-08-11T15:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.703+0000 7f6052973b80 10 client.0 _ll_register_callbacks cb 0x55e393768a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.704+0000 7f603cd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875630 0x55e393884ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.704+0000 7f603cd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875630 0x55e393884ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:49020/0 (socket says 172.21.15.32:49020) 2023-08-11T15:53:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.704+0000 7f603cd9b700 1 -- 172.21.15.32:0/3919498007 learned_addr learned my addr 172.21.15.32:0/3919498007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.704+0000 7f602ffff700 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875ff0 0x55e393884ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.705+0000 7f603d59c700 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 0x55e39387eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.705+0000 7f603cd9b700 1 -- 172.21.15.32:0/3919498007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875ff0 msgr2=0x55e393884ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.705+0000 7f603cd9b700 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875ff0 0x55e393884ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.705+0000 7f603cd9b700 1 -- 172.21.15.32:0/3919498007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 msgr2=0x55e39387eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.705+0000 7f603cd9b700 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 0x55e39387eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.705+0000 7f603cd9b700 1 -- 172.21.15.32:0/3919498007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603810c7e0 con 0x55e393875630 2023-08-11T15:53:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.709+0000 7f603cd9b700 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875630 0x55e393884ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f60280095c0 tx=0x7f6028003b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2da360cbcdf4ac server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.709+0000 7f6017fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.712+0000 7f6017fff700 1 -- 172.21.15.32:0/3919498007 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028003dd0 con 0x55e393875630 2023-08-11T15:53:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.712+0000 7f6017fff700 1 -- 172.21.15.32:0/3919498007 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f6028005280 con 0x55e393875630 2023-08-11T15:53:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:54 smithi134 ceph-mon[130447]: pgmap v226: 129 pgs: 129 active+clean; 79 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2023-08-11T15:53:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.718+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3938687e0 con 0x55e393875630 2023-08-11T15:53:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.722+0000 7f6017fff700 1 -- 172.21.15.32:0/3919498007 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6028005440 con 0x55e393875630 2023-08-11T15:53:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.724+0000 7f6052973b80 10 client.25982 fetch_fsmap learned FSMap version 31 2023-08-11T15:53:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.724+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e3935f3d80 con 0x55e393875630 2023-08-11T15:53:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.732+0000 7f6017fff700 1 -- 172.21.15.32:0/3919498007 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3938687e0 con 0x55e393875630 2023-08-11T15:53:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.733+0000 7f6017fff700 1 -- 172.21.15.32:0/3919498007 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6028034070 con 0x55e393875630 2023-08-11T15:53:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.734+0000 7f6052973b80 10 client.25982 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:53:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.734+0000 7f6052973b80 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.734+0000 7f6052973b80 20 client.25982 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.734+0000 7f6052973b80 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e39388ea30 0x55e393890e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.734+0000 7f6052973b80 4 client.25982 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:53:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.734+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e393844480 con 0x55e39388ea30 2023-08-11T15:53:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.735+0000 7f602ffff700 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e39388ea30 0x55e393890e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.738+0000 7f602ffff700 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e39388ea30 0x55e393890e20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab4fbb0551d7629 server_cookie=8d7132c94fb82760 in_seq=0 out_seq=0 2023-08-11T15:53:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.740+0000 7f6017fff700 10 client.25982 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.740+0000 7f6017fff700 1 -- 172.21.15.32:0/3919498007 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e393844480 con 0x55e39388ea30 2023-08-11T15:53:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.740+0000 7f6017fff700 10 client.25982 handle_command_reply: tid=0 2023-08-11T15:53:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.741+0000 7f6052973b80 1 client.25982 shutdown 2023-08-11T15:53:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875630 msgr2=0x55e393884ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875630 0x55e393884ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f60280095c0 tx=0x7f6028003b70 comp rx=0 tx=0).stop 2023-08-11T15:53:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 shutdown_connections 2023-08-11T15:53:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e39388ea30 0x55e393890e20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e39387ae80 0x55e39387eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e393875630 0x55e393884ab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 --2- 172.21.15.32:0/3919498007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875ff0 0x55e393884ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 >> 172.21.15.32:0/3919498007 conn(0x55e3937a28f0 msgr2=0x55e39386f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6017fff700 0 client.25982 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 shutdown_connections 2023-08-11T15:53:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 1 -- 172.21.15.32:0/3919498007 wait complete. 2023-08-11T15:53:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.742+0000 7f6052973b80 20 client.25982 trim_cache size 0 max 16384 2023-08-11T15:53:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 Processor -- start 2023-08-11T15:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 -- start start 2023-08-11T15:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e39387ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e39386fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393879f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e393895bd0 con 0x55e393875ff0 2023-08-11T15:53:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e393871db0 con 0x55e39387ae80 2023-08-11T15:53:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.759+0000 7f6052973b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e39386aea0 con 0x55e393875630 2023-08-11T15:53:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393879f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e39387ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e39387ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:43958/0 (socket says 172.21.15.32:43958) 2023-08-11T15:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393879f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:49024/0 (socket says 172.21.15.32:49024) 2023-08-11T15:53:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 -- 172.21.15.32:0/237011460 learned_addr learned my addr 172.21.15.32:0/237011460 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f602ffff700 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e39386fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 -- 172.21.15.32:0/237011460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 msgr2=0x55e393879f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393879f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 -- 172.21.15.32:0/237011460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 msgr2=0x55e39386fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e39386fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 -- 172.21.15.32:0/237011460 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3935dcc00 con 0x55e393875630 2023-08-11T15:53:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603d59c700 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393879f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:53:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f603cd9b700 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e39387ca00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36430281f9e2f5b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.760+0000 7f601f7fe700 1 -- 172.21.15.32:0/237011460 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028005280 con 0x55e393875630 2023-08-11T15:53:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f601f7fe700 1 -- 172.21.15.32:0/237011460 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6028022df0 con 0x55e393875630 2023-08-11T15:53:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f601f7fe700 1 -- 172.21.15.32:0/237011460 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800f6c0 con 0x55e393875630 2023-08-11T15:53:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 -- 172.21.15.32:0/237011460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 msgr2=0x55e39387ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e39387ca00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028001e50 comp rx=0 tx=0).stop 2023-08-11T15:53:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 -- 172.21.15.32:0/237011460 shutdown_connections 2023-08-11T15:53:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393879f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e39387ca00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 --2- 172.21.15.32:0/237011460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e39386fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 -- 172.21.15.32:0/237011460 >> 172.21.15.32:0/237011460 conn(0x55e3937a28f0 msgr2=0x55e393878fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 -- 172.21.15.32:0/237011460 shutdown_connections 2023-08-11T15:53:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.761+0000 7f6052973b80 1 -- 172.21.15.32:0/237011460 wait complete. 2023-08-11T15:53:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.762+0000 7f6052973b80 1 -- start start 2023-08-11T15:53:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.762+0000 7f6052973b80 1 Processor -- start 2023-08-11T15:53:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6052973b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e3938823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6052973b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e393882900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6052973b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393882e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6052973b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3938e1660 con 0x55e393875ff0 2023-08-11T15:53:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6052973b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3938e17a0 con 0x55e39387ae80 2023-08-11T15:53:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6052973b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3938e18e0 con 0x55e393875630 2023-08-11T15:53:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6052973b80 10 client.0 _ll_register_callbacks cb 0x7f6028009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:53:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393882e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f602ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e393882900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393882e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:49028/0 (socket says 172.21.15.32:49028) 2023-08-11T15:53:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 -- 172.21.15.32:0/2187729042 learned_addr learned my addr 172.21.15.32:0/2187729042 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603cd9b700 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e3938823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 -- 172.21.15.32:0/2187729042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 msgr2=0x55e3938823c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e3938823c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 -- 172.21.15.32:0/2187729042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 msgr2=0x55e393882900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e393882900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 -- 172.21.15.32:0/2187729042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60381bafb0 con 0x55e39387ae80 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f603d59c700 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393882e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6030095dd0 tx=0x7f6030099b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ceeb74724cb00e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:53:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.763+0000 7f6017fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:53:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6017fff700 1 -- 172.21.15.32:0/2187729042 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300997c0 con 0x55e39387ae80 2023-08-11T15:53:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f602801e040 con 0x55e39387ae80 2023-08-11T15:53:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6017fff700 1 -- 172.21.15.32:0/2187729042 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f60300a1170 con 0x55e39387ae80 2023-08-11T15:53:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6017fff700 1 -- 172.21.15.32:0/2187729042 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6030098850 con 0x55e39387ae80 2023-08-11T15:53:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 10 client.25988 fetch_fsmap learned FSMap version 31 2023-08-11T15:53:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e3935f3d80 con 0x55e39387ae80 2023-08-11T15:53:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6017fff700 1 -- 172.21.15.32:0/2187729042 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60300c6080 con 0x55e39387ae80 2023-08-11T15:53:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6017fff700 1 -- 172.21.15.32:0/2187729042 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f60300a0ce0 con 0x55e39387ae80 2023-08-11T15:53:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 10 client.25988 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:53:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:53:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 20 client.25988 populate_metadata read hostname 'smithi032' 2023-08-11T15:53:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e3938e3020 0x55e3938e5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:53:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 4 client.25988 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:53:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3937c07c0 con 0x55e3938e3020 2023-08-11T15:53:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.764+0000 7f603cd9b700 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e3938e3020 0x55e3938e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:53:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.765+0000 7f603cd9b700 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e3938e3020 0x55e3938e5440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a855cb5a8ff72f server_cookie=ce80a8af8785f58a in_seq=0 out_seq=0 2023-08-11T15:53:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:54.765+0000 7f6017fff700 10 client.25988 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:54.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 818288 11.32 MB/sec execute 250 sec latency 115.447 ms 2023-08-11T15:53:54.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:54 smithi032 ceph-mon[97103]: pgmap v226: 129 pgs: 129 active+clean; 79 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2023-08-11T15:53:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6017fff700 1 -- 172.21.15.32:0/2187729042 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3937c07c0 con 0x55e3938e3020 2023-08-11T15:53:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6017fff700 10 client.25988 handle_command_reply: tid=0 2023-08-11T15:53:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 client.25988 shutdown 2023-08-11T15:53:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 msgr2=0x55e393882e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393882e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6030095dd0 tx=0x7f6030099b30 comp rx=0 tx=0).stop 2023-08-11T15:53:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 shutdown_connections 2023-08-11T15:53:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e3938e3020 0x55e3938e5440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e39387ae80 0x55e393882e40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e393875630 0x55e3938823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 --2- 172.21.15.32:0/2187729042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e393875ff0 0x55e393882900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 >> 172.21.15.32:0/2187729042 conn(0x55e3937a28f0 msgr2=0x55e393879f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6017fff700 0 client.25988 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:53:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.083+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 shutdown_connections 2023-08-11T15:53:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.084+0000 7f6052973b80 1 -- 172.21.15.32:0/2187729042 wait complete. 2023-08-11T15:53:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.084+0000 7f6052973b80 20 client.25988 trim_cache size 0 max 16384 2023-08-11T15:53:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:53:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:53:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:53:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:53:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f60200a02d0 msgr2=0x7f60200a2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f60200a02d0 0x7f60200a2790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6034005fd0 tx=0x7f6034009510 comp rx=0 tx=0).stop 2023-08-11T15:53:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 msgr2=0x7f603811cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:53:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 0x7f603811cd30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6030007ee0 tx=0x7f603000d3b0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 shutdown_connections 2023-08-11T15:53:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f60200a02d0 0x7f60200a2790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60381121a0 0x7f603811cd30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6038068160 0x7f6038117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.085+0000 7f603efff700 1 --2- 172.21.15.32:0/3291033177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6038107e60 0x7f6038117750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:53:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.086+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 >> 172.21.15.32:0/3291033177 conn(0x7f603806ce20 msgr2=0x7f603810af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:53:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.086+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 shutdown_connections 2023-08-11T15:53:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:53:55.086+0000 7f603efff700 1 -- 172.21.15.32:0/3291033177 wait complete. 2023-08-11T15:53:55.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 820221 11.32 MB/sec execute 251 sec latency 123.726 ms 2023-08-11T15:53:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:56 smithi134 ceph-mon[130447]: pgmap v227: 129 pgs: 129 active+clean; 79 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2023-08-11T15:53:56.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 821961 11.31 MB/sec execute 252 sec latency 13.789 ms 2023-08-11T15:53:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:56 smithi032 ceph-mon[97103]: pgmap v227: 129 pgs: 129 active+clean; 79 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2023-08-11T15:53:56.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:56 smithi192 ceph-mon[130806]: pgmap v227: 129 pgs: 129 active+clean; 79 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2023-08-11T15:53:57.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 824303 11.32 MB/sec execute 253 sec latency 73.330 ms 2023-08-11T15:53:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:57 smithi032 ceph-mon[97103]: pgmap v228: 129 pgs: 129 active+clean; 67 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2023-08-11T15:53:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:57 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:53:58.340 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:58 smithi192 ceph-mon[130806]: pgmap v228: 129 pgs: 129 active+clean; 67 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2023-08-11T15:53:58.341 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:58 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:53:58.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:57 smithi134 ceph-mon[130447]: pgmap v228: 129 pgs: 129 active+clean; 67 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2023-08-11T15:53:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:57 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:53:58.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 826450 11.32 MB/sec execute 254 sec latency 25.551 ms 2023-08-11T15:53:59.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:59 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:53:59.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:59 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:53:59.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:53:59 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:53:59.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:59 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:53:59.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:59 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:53:59.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:53:59 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:53:59.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:59 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:53:59.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:59 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:53:59.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:53:59 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:53:59.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 828217 11.31 MB/sec execute 255 sec latency 61.833 ms 2023-08-11T15:54:00.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:00 smithi032 ceph-mon[97103]: pgmap v229: 129 pgs: 129 active+clean; 69 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2023-08-11T15:54:00.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:00 smithi192 ceph-mon[130806]: pgmap v229: 129 pgs: 129 active+clean; 69 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2023-08-11T15:54:00.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:00 smithi134 ceph-mon[130447]: pgmap v229: 129 pgs: 129 active+clean; 69 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2023-08-11T15:54:00.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 830564 11.31 MB/sec execute 256 sec latency 72.787 ms 2023-08-11T15:54:01.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 830970 11.29 MB/sec execute 257 sec latency 64.668 ms 2023-08-11T15:54:02.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:02 smithi192 ceph-mon[130806]: pgmap v230: 129 pgs: 129 active+clean; 69 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2023-08-11T15:54:02.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:02 smithi134 ceph-mon[130447]: pgmap v230: 129 pgs: 129 active+clean; 69 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2023-08-11T15:54:02.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 832479 11.27 MB/sec execute 258 sec latency 59.501 ms 2023-08-11T15:54:02.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:02 smithi032 ceph-mon[97103]: pgmap v230: 129 pgs: 129 active+clean; 69 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2023-08-11T15:54:03.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 833243 11.23 MB/sec execute 259 sec latency 97.305 ms 2023-08-11T15:54:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:04 smithi032 ceph-mon[97103]: osdmap e101: 12 total, 12 up, 12 in 2023-08-11T15:54:04.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:04 smithi032 ceph-mon[97103]: pgmap v232: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 435 op/s 2023-08-11T15:54:04.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:04 smithi192 ceph-mon[130806]: osdmap e101: 12 total, 12 up, 12 in 2023-08-11T15:54:04.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:04 smithi192 ceph-mon[130806]: pgmap v232: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 435 op/s 2023-08-11T15:54:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:04 smithi134 ceph-mon[130447]: osdmap e101: 12 total, 12 up, 12 in 2023-08-11T15:54:04.472 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:04 smithi134 ceph-mon[130447]: pgmap v232: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 435 op/s 2023-08-11T15:54:04.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 833748 11.21 MB/sec execute 260 sec latency 61.923 ms 2023-08-11T15:54:05.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 834112 11.17 MB/sec execute 261 sec latency 145.950 ms 2023-08-11T15:54:06.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:06 smithi134 ceph-mon[130447]: pgmap v233: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 435 op/s 2023-08-11T15:54:06.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 834542 11.15 MB/sec execute 262 sec latency 93.331 ms 2023-08-11T15:54:06.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:06 smithi032 ceph-mon[97103]: pgmap v233: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 435 op/s 2023-08-11T15:54:06.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:06 smithi192 ceph-mon[130806]: pgmap v233: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 435 op/s 2023-08-11T15:54:07.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 834912 11.10 MB/sec execute 263 sec latency 68.554 ms 2023-08-11T15:54:08.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:08 smithi032 ceph-mon[97103]: pgmap v234: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-08-11T15:54:08.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:08 smithi192 ceph-mon[130806]: pgmap v234: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-08-11T15:54:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.338+0000 7f79dd3ff700 1 -- 172.21.15.32:0/2976844121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d811a100 msgr2=0x7f79d811c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.338+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/2976844121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d811a100 0x7f79d811c4f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009990 tx=0x7f79d0009ca0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.339+0000 7f79dd3ff700 1 -- 172.21.15.32:0/2976844121 shutdown_connections 2023-08-11T15:54:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.339+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/2976844121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d810f070 0x7f79d8107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.339+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/2976844121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d811a100 0x7f79d811c4f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.339+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/2976844121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79d810f4e0 0x7f79d8107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.339+0000 7f79dd3ff700 1 -- 172.21.15.32:0/2976844121 >> 172.21.15.32:0/2976844121 conn(0x7f79d806c9e0 msgr2=0x7f79d806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.341+0000 7f79dd3ff700 1 -- 172.21.15.32:0/2976844121 shutdown_connections 2023-08-11T15:54:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.345+0000 7f79dd3ff700 1 -- 172.21.15.32:0/2976844121 wait complete. 2023-08-11T15:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.346+0000 7f79dd3ff700 1 Processor -- start 2023-08-11T15:54:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.347+0000 7f79dd3ff700 1 -- start start 2023-08-11T15:54:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.347+0000 7f79dd3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d810f070 0x7f79d81a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.347+0000 7f79dd3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d810f4e0 0x7f79d81a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.347+0000 7f79dd3ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79d811a100 0x7f79d81aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.348+0000 7f79dd3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d81aa950 con 0x7f79d810f4e0 2023-08-11T15:54:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.348+0000 7f79dd3ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d81aaac0 con 0x7f79d811a100 2023-08-11T15:54:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.348+0000 7f79dd3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d81aac80 con 0x7f79d810f070 2023-08-11T15:54:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.348+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79d811a100 0x7f79d81aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.348+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79d811a100 0x7f79d81aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55990/0 (socket says 172.21.15.32:55990) 2023-08-11T15:54:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.348+0000 7f79d77fe700 1 -- 172.21.15.32:0/178048571 learned_addr learned my addr 172.21.15.32:0/178048571 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.349+0000 7f79d67fc700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d810f4e0 0x7f79d81a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.349+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d810f070 0x7f79d81a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.350+0000 7f79d67fc700 1 -- 172.21.15.32:0/178048571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d810f070 msgr2=0x7f79d81a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.350+0000 7f79d67fc700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d810f070 0x7f79d81a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.350+0000 7f79d67fc700 1 -- 172.21.15.32:0/178048571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79d811a100 msgr2=0x7f79d81aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.350+0000 7f79d67fc700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79d811a100 0x7f79d81aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.350+0000 7f79d67fc700 1 -- 172.21.15.32:0/178048571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d00095f0 con 0x7f79d810f4e0 2023-08-11T15:54:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.350+0000 7f79d67fc700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d810f4e0 0x7f79d81a5ce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f79c8007e90 tx=0x7f79c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.351+0000 7f79cffff700 1 -- 172.21.15.32:0/178048571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c800ddd0 con 0x7f79d810f4e0 2023-08-11T15:54:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.351+0000 7f79cffff700 1 -- 172.21.15.32:0/178048571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c800f040 con 0x7f79d810f4e0 2023-08-11T15:54:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.351+0000 7f79cffff700 1 -- 172.21.15.32:0/178048571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8005980 con 0x7f79d810f4e0 2023-08-11T15:54:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.351+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d81aaf60 con 0x7f79d810f4e0 2023-08-11T15:54:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.352+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d81b2810 con 0x7f79d810f4e0 2023-08-11T15:54:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.358+0000 7f79cffff700 1 -- 172.21.15.32:0/178048571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f79c8004ad0 con 0x7f79d810f4e0 2023-08-11T15:54:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.360+0000 7f79cffff700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f79c40a02d0 0x7f79c40a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.361+0000 7f79cffff700 1 -- 172.21.15.32:0/178048571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f79c80cb290 con 0x7f79d810f4e0 2023-08-11T15:54:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.366+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f79c40a02d0 0x7f79c40a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.367+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f79c40a02d0 0x7f79c40a2790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79c0006fd0 tx=0x7f79c0008040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.368+0000 7f79f0ca1b80 1 Processor -- start 2023-08-11T15:54:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.370+0000 7f79f0ca1b80 1 -- start start 2023-08-11T15:54:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.370+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b2f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.370+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b29ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.370+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.370+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5a825d0 con 0x55a9f5b2ee80 2023-08-11T15:54:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.370+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5a7fef0 con 0x55a9f5b29ff0 2023-08-11T15:54:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.370+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5a7b7e0 con 0x55a9f5b29630 2023-08-11T15:54:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b35c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:56000/0 (socket says 172.21.15.32:56000) 2023-08-11T15:54:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 -- 172.21.15.32:0/2530492675 learned_addr learned my addr 172.21.15.32:0/2530492675 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 -- 172.21.15.32:0/2530492675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 msgr2=0x55a9f5b29ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T15:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 --2- 172.21.15.32:0/2530492675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b29ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 -- 172.21.15.32:0/2530492675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 msgr2=0x55a9f5b2f260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 --2- 172.21.15.32:0/2530492675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b2f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.371+0000 7f79d77fe700 1 -- 172.21.15.32:0/2530492675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f58fb5d0 con 0x55a9f5b29ff0 2023-08-11T15:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.372+0000 7f79d77fe700 1 --2- 172.21.15.32:0/2530492675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b35c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009f10 tx=0x7f79d0009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5da8d8be79e5632 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.372+0000 7f79cd7fa700 1 -- 172.21.15.32:0/2530492675 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d001d030 con 0x55a9f5b29ff0 2023-08-11T15:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.372+0000 7f79cd7fa700 1 -- 172.21.15.32:0/2530492675 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d001e040 con 0x55a9f5b29ff0 2023-08-11T15:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.372+0000 7f79cd7fa700 1 -- 172.21.15.32:0/2530492675 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0005650 con 0x55a9f5b29ff0 2023-08-11T15:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.373+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2530492675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 msgr2=0x55a9f5b35c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.373+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2530492675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b35c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009f10 tx=0x7f79d0009f40 comp rx=0 tx=0).stop 2023-08-11T15:54:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.374+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2530492675 shutdown_connections 2023-08-11T15:54:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.374+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2530492675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b2f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.374+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2530492675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b35c10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.374+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2530492675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b29ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.374+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2530492675 >> 172.21.15.32:0/2530492675 conn(0x55a9f5a568f0 msgr2=0x55a9f5b23e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.377+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2530492675 shutdown_connections 2023-08-11T15:54:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.378+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2530492675 wait complete. 2023-08-11T15:54:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.378+0000 7f79f0ca1b80 1 -- start start 2023-08-11T15:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.379+0000 7f79f0ca1b80 1 Processor -- start 2023-08-11T15:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.379+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b35b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.379+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b32df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.379+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.379+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5a19a60 con 0x55a9f5b2ee80 2023-08-11T15:54:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.379+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5a19760 con 0x55a9f5b29ff0 2023-08-11T15:54:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.379+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f58fd8c0 con 0x55a9f5b29630 2023-08-11T15:54:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.380+0000 7f79f0ca1b80 10 client.0 _ll_register_callbacks cb 0x55a9f5a1ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.380+0000 7f79d6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b35b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.380+0000 7f79d6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b35b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:48532/0 (socket says 172.21.15.32:48532) 2023-08-11T15:54:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.380+0000 7f79d6ffd700 1 -- 172.21.15.32:0/4060978680 learned_addr learned my addr 172.21.15.32:0/4060978680 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d77fe700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d6ffd700 1 -- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 msgr2=0x55a9f5b32df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d67fc700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b32df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b32df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d6ffd700 1 -- 172.21.15.32:0/4060978680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 msgr2=0x55a9f5b33400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b33400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d6ffd700 1 -- 172.21.15.32:0/4060978680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d811ce20 con 0x55a9f5b29630 2023-08-11T15:54:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d77fe700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b33400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:54:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.381+0000 7f79d67fc700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:54:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.382+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b35b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f79c000dca0 tx=0x7f79c0004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a8008f6edb4a8d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.382+0000 7f79be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:54:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.382+0000 7f79be7fc700 1 -- 172.21.15.32:0/4060978680 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0014bd0 con 0x55a9f5b29630 2023-08-11T15:54:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.382+0000 7f79be7fc700 1 -- 172.21.15.32:0/4060978680 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f79c0014d30 con 0x55a9f5b29630 2023-08-11T15:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.382+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f5b1c7e0 con 0x55a9f5b29630 2023-08-11T15:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.383+0000 7f79be7fc700 1 -- 172.21.15.32:0/4060978680 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79c001f400 con 0x55a9f5b29630 2023-08-11T15:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.383+0000 7f79f0ca1b80 10 client.26002 fetch_fsmap learned FSMap version 31 2023-08-11T15:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.383+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a9f58a7d80 con 0x55a9f5b29630 2023-08-11T15:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.383+0000 7f79be7fc700 1 -- 172.21.15.32:0/4060978680 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9f5b1c7e0 con 0x55a9f5b29630 2023-08-11T15:54:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.384+0000 7f79be7fc700 1 -- 172.21.15.32:0/4060978680 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79c001e400 con 0x55a9f5b29630 2023-08-11T15:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.384+0000 7f79f0ca1b80 10 client.26002 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.384+0000 7f79f0ca1b80 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.384+0000 7f79f0ca1b80 20 client.26002 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.384+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5b53ff0 0x55a9f5b56410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.384+0000 7f79f0ca1b80 4 client.26002 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:54:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.384+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f5af8480 con 0x55a9f5b53ff0 2023-08-11T15:54:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.385+0000 7f79d67fc700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5b53ff0 0x55a9f5b56410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.386+0000 7f79d67fc700 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5b53ff0 0x55a9f5b56410 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4444fb2c8879acfd server_cookie=2a5ff6acdaf883c6 in_seq=0 out_seq=0 2023-08-11T15:54:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.386+0000 7f79be7fc700 10 client.26002 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.387+0000 7f79be7fc700 1 -- 172.21.15.32:0/4060978680 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a9f5af8480 con 0x55a9f5b53ff0 2023-08-11T15:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.387+0000 7f79be7fc700 10 client.26002 handle_command_reply: tid=0 2023-08-11T15:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.388+0000 7f79f0ca1b80 1 client.26002 shutdown 2023-08-11T15:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 msgr2=0x55a9f5b35b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b35b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f79c000dca0 tx=0x7f79c0004860 comp rx=0 tx=0).stop 2023-08-11T15:54:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 shutdown_connections 2023-08-11T15:54:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5b53ff0 0x55a9f5b56410 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b33400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b35b40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4060978680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.390+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 >> 172.21.15.32:0/4060978680 conn(0x55a9f5a568f0 msgr2=0x55a9f5b235d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.391+0000 7f79be7fc700 0 client.26002 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.394+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 shutdown_connections 2023-08-11T15:54:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.394+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4060978680 wait complete. 2023-08-11T15:54:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.395+0000 7f79f0ca1b80 20 client.26002 trim_cache size 0 max 16384 2023-08-11T15:54:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.425+0000 7f79f0ca1b80 1 Processor -- start 2023-08-11T15:54:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79f0ca1b80 1 -- start start 2023-08-11T15:54:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b23d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b35600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5b25db0 con 0x55a9f5b29ff0 2023-08-11T15:54:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5b516e0 con 0x55a9f5b2ee80 2023-08-11T15:54:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5b2eac0 con 0x55a9f5b29630 2023-08-11T15:54:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.426+0000 7f79d6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:48542/0 (socket says 172.21.15.32:48542) 2023-08-11T15:54:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d6ffd700 1 -- 172.21.15.32:0/4045253952 learned_addr learned my addr 172.21.15.32:0/4045253952 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d77fe700 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d67fc700 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b23d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d77fe700 1 -- 172.21.15.32:0/4045253952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 msgr2=0x55a9f5b2e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d77fe700 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b2e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d77fe700 1 -- 172.21.15.32:0/4045253952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b29ff0 msgr2=0x55a9f5b23d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d77fe700 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b23d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.427+0000 7f79d77fe700 1 -- 172.21.15.32:0/4045253952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f5890c00 con 0x55a9f5b2ee80 2023-08-11T15:54:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.428+0000 7f79d77fe700 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b35600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005950 tx=0x7f79d000b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687cc4878ca3d917 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.428+0000 7f79cdffb700 1 -- 172.21.15.32:0/4045253952 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d001d050 con 0x55a9f5b2ee80 2023-08-11T15:54:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.428+0000 7f79cdffb700 1 -- 172.21.15.32:0/4045253952 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d000ba90 con 0x55a9f5b2ee80 2023-08-11T15:54:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.428+0000 7f79cdffb700 1 -- 172.21.15.32:0/4045253952 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0023460 con 0x55a9f5b2ee80 2023-08-11T15:54:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.429+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4045253952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b2ee80 msgr2=0x55a9f5b35600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.429+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b35600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005950 tx=0x7f79d000b540 comp rx=0 tx=0).stop 2023-08-11T15:54:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.430+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4045253952 shutdown_connections 2023-08-11T15:54:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.430+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b2ee80 0x55a9f5b35600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.430+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b2e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.430+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/4045253952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b23d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.430+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4045253952 >> 172.21.15.32:0/4045253952 conn(0x55a9f5a568f0 msgr2=0x55a9f5b2de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.430+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4045253952 shutdown_connections 2023-08-11T15:54:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.430+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/4045253952 wait complete. 2023-08-11T15:54:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.431+0000 7f79f0ca1b80 1 -- start start 2023-08-11T15:54:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.431+0000 7f79f0ca1b80 1 Processor -- start 2023-08-11T15:54:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b49360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79f0ca1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5ba6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5ba71e0 con 0x55a9f5b2ee80 2023-08-11T15:54:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5ba7320 con 0x55a9f5b29ff0 2023-08-11T15:54:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79f0ca1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5ba74b0 con 0x55a9f5b29630 2023-08-11T15:54:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79f0ca1b80 10 client.0 _ll_register_callbacks cb 0x7f79c000d670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.432+0000 7f79d6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b49360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b49360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:48552/0 (socket says 172.21.15.32:48552) 2023-08-11T15:54:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d6ffd700 1 -- 172.21.15.32:0/2760840102 learned_addr learned my addr 172.21.15.32:0/2760840102 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d77fe700 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5ba6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d6ffd700 1 -- 172.21.15.32:0/2760840102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 msgr2=0x55a9f5b498a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T15:54:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b498a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d6ffd700 1 -- 172.21.15.32:0/2760840102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 msgr2=0x55a9f5ba6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5ba6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.433+0000 7f79d6ffd700 1 -- 172.21.15.32:0/2760840102 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9f58fb5d0 con 0x55a9f5b29630 2023-08-11T15:54:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.438+0000 7f79d6ffd700 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b49360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f79c001e7c0 tx=0x7f79c0009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5612c3ccfb9fdf25 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.438+0000 7f79be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:54:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.438+0000 7f79be7fc700 1 -- 172.21.15.32:0/2760840102 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c001e5f0 con 0x55a9f5b29630 2023-08-11T15:54:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.438+0000 7f79be7fc700 1 -- 172.21.15.32:0/2760840102 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9483+0+0 (secure 0 0 0) 0x7f79c0014b50 con 0x55a9f5b29630 2023-08-11T15:54:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.439+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79d001e040 con 0x55a9f5b29630 2023-08-11T15:54:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.439+0000 7f79be7fc700 1 -- 172.21.15.32:0/2760840102 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79c0014d10 con 0x55a9f5b29630 2023-08-11T15:54:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.439+0000 7f79f0ca1b80 10 client.26008 fetch_fsmap learned FSMap version 31 2023-08-11T15:54:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.439+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a9f58ce1f0 con 0x55a9f5b29630 2023-08-11T15:54:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79be7fc700 1 -- 172.21.15.32:0/2760840102 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c003c030 con 0x55a9f5b29630 2023-08-11T15:54:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79be7fc700 1 -- 172.21.15.32:0/2760840102 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79c0036060 con 0x55a9f5b29630 2023-08-11T15:54:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79f0ca1b80 10 client.26008 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:54:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79f0ca1b80 10 client.26008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79f0ca1b80 20 client.26008 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5ba8bf0 0x55a9f5bab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79f0ca1b80 4 client.26008 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:54:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.440+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9f5a747c0 con 0x55a9f5ba8bf0 2023-08-11T15:54:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.442+0000 7f79d67fc700 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5ba8bf0 0x55a9f5bab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.443+0000 7f79d67fc700 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5ba8bf0 0x55a9f5bab010 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690f5b9a3a51fc86 server_cookie=a3f316e048fbf21c in_seq=0 out_seq=0 2023-08-11T15:54:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.443+0000 7f79be7fc700 10 client.26008 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:08.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:08 smithi134 ceph-mon[130447]: pgmap v234: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-08-11T15:54:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.757+0000 7f79be7fc700 1 -- 172.21.15.32:0/2760840102 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9f5a747c0 con 0x55a9f5ba8bf0 2023-08-11T15:54:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.757+0000 7f79be7fc700 10 client.26008 handle_command_reply: tid=0 2023-08-11T15:54:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.757+0000 7f79f0ca1b80 1 client.26008 shutdown 2023-08-11T15:54:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.758+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 msgr2=0x55a9f5b49360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.758+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b49360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f79c001e7c0 tx=0x7f79c0009710 comp rx=0 tx=0).stop 2023-08-11T15:54:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.759+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 shutdown_connections 2023-08-11T15:54:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.759+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55a9f5ba8bf0 0x55a9f5bab010 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.759+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9f5b2ee80 0x55a9f5ba6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.759+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9f5b29630 0x55a9f5b49360 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.759+0000 7f79f0ca1b80 1 --2- 172.21.15.32:0/2760840102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9f5b29ff0 0x55a9f5b498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.759+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 >> 172.21.15.32:0/2760840102 conn(0x55a9f5a568f0 msgr2=0x55a9f5b28620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.760+0000 7f79be7fc700 0 client.26008 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.760+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 shutdown_connections 2023-08-11T15:54:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.760+0000 7f79f0ca1b80 1 -- 172.21.15.32:0/2760840102 wait complete. 2023-08-11T15:54:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.760+0000 7f79f0ca1b80 20 client.26008 trim_cache size 0 max 16384 2023-08-11T15:54:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:54:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:54:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:54:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:54:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.763+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f79c40a02d0 msgr2=0x7f79c40a2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.763+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f79c40a02d0 0x7f79c40a2790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79c0006fd0 tx=0x7f79c0008040 comp rx=0 tx=0).stop 2023-08-11T15:54:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.763+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d810f4e0 msgr2=0x7f79d81a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.763+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d810f4e0 0x7f79d81a5ce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f79c8007e90 tx=0x7f79c800d3b0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.765+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 shutdown_connections 2023-08-11T15:54:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.765+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f79c40a02d0 0x7f79c40a2790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.765+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79d811a100 0x7f79d81aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.765+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d810f070 0x7f79d81a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.765+0000 7f79dd3ff700 1 --2- 172.21.15.32:0/178048571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d810f4e0 0x7f79d81a5ce0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.765+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 >> 172.21.15.32:0/178048571 conn(0x7f79d806c9e0 msgr2=0x7f79d81180f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.766+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 shutdown_connections 2023-08-11T15:54:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:08.766+0000 7f79dd3ff700 1 -- 172.21.15.32:0/178048571 wait complete. 2023-08-11T15:54:08.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 835308 11.07 MB/sec execute 264 sec latency 178.626 ms 2023-08-11T15:54:09.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 836953 11.05 MB/sec execute 265 sec latency 31.451 ms 2023-08-11T15:54:10.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:10 smithi134 ceph-mon[130447]: pgmap v235: 129 pgs: 11 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 100 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2023-08-11T15:54:10.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 838590 11.05 MB/sec execute 266 sec latency 54.830 ms 2023-08-11T15:54:10.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:10 smithi032 ceph-mon[97103]: pgmap v235: 129 pgs: 11 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 100 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2023-08-11T15:54:10.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:10 smithi192 ceph-mon[130806]: pgmap v235: 129 pgs: 11 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 100 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2023-08-11T15:54:11.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 841208 11.06 MB/sec execute 267 sec latency 22.524 ms 2023-08-11T15:54:12.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:12 smithi134 ceph-mon[130447]: pgmap v236: 129 pgs: 11 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 100 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2023-08-11T15:54:12.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 842967 11.06 MB/sec execute 268 sec latency 41.089 ms 2023-08-11T15:54:12.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:12 smithi032 ceph-mon[97103]: pgmap v236: 129 pgs: 11 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 100 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2023-08-11T15:54:12.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:12 smithi192 ceph-mon[130806]: pgmap v236: 129 pgs: 11 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 100 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2023-08-11T15:54:13.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 845475 11.07 MB/sec execute 269 sec latency 13.022 ms 2023-08-11T15:54:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:14 smithi134 ceph-mon[130447]: pgmap v237: 129 pgs: 129 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 284 op/s 2023-08-11T15:54:14.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 847468 11.06 MB/sec execute 270 sec latency 32.522 ms 2023-08-11T15:54:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:14 smithi032 ceph-mon[97103]: pgmap v237: 129 pgs: 129 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 284 op/s 2023-08-11T15:54:14.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:14 smithi192 ceph-mon[130806]: pgmap v237: 129 pgs: 129 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 284 op/s 2023-08-11T15:54:15.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 849230 11.06 MB/sec execute 271 sec latency 89.043 ms 2023-08-11T15:54:16.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:16 smithi134 ceph-mon[130447]: pgmap v238: 129 pgs: 129 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.3 MiB/s wr, 244 op/s 2023-08-11T15:54:16.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 851340 11.06 MB/sec execute 272 sec latency 27.293 ms 2023-08-11T15:54:16.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:16 smithi032 ceph-mon[97103]: pgmap v238: 129 pgs: 129 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.3 MiB/s wr, 244 op/s 2023-08-11T15:54:16.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:16 smithi192 ceph-mon[130806]: pgmap v238: 129 pgs: 129 active+clean; 66 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.3 MiB/s wr, 244 op/s 2023-08-11T15:54:17.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 852901 11.05 MB/sec execute 273 sec latency 35.307 ms 2023-08-11T15:54:18.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:18 smithi032 ceph-mon[97103]: osdmap e102: 12 total, 12 up, 12 in 2023-08-11T15:54:18.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:18 smithi032 ceph-mon[97103]: pgmap v240: 129 pgs: 129 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2023-08-11T15:54:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:18 smithi192 ceph-mon[130806]: osdmap e102: 12 total, 12 up, 12 in 2023-08-11T15:54:18.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:18 smithi192 ceph-mon[130806]: pgmap v240: 129 pgs: 129 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2023-08-11T15:54:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:18 smithi134 ceph-mon[130447]: osdmap e102: 12 total, 12 up, 12 in 2023-08-11T15:54:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:18 smithi134 ceph-mon[130447]: pgmap v240: 129 pgs: 129 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2023-08-11T15:54:18.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 855466 11.06 MB/sec execute 274 sec latency 14.002 ms 2023-08-11T15:54:19.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 857038 11.06 MB/sec execute 275 sec latency 130.427 ms 2023-08-11T15:54:20.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:20 smithi134 ceph-mon[130447]: pgmap v241: 129 pgs: 129 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-08-11T15:54:20.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 858633 11.04 MB/sec execute 276 sec latency 213.599 ms 2023-08-11T15:54:20.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:20 smithi032 ceph-mon[97103]: pgmap v241: 129 pgs: 129 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-08-11T15:54:20.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:20 smithi192 ceph-mon[130806]: pgmap v241: 129 pgs: 129 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-08-11T15:54:21.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 860372 11.03 MB/sec execute 277 sec latency 53.659 ms 2023-08-11T15:54:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.934+0000 7f75365ff700 1 -- 172.21.15.32:0/2677708289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301068c0 msgr2=0x7f7530106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.934+0000 7f75365ff700 1 --2- 172.21.15.32:0/2677708289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301068c0 0x7f7530106ca0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7520009990 tx=0x7f7520009ca0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.934+0000 7f75365ff700 1 -- 172.21.15.32:0/2677708289 shutdown_connections 2023-08-11T15:54:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.934+0000 7f75365ff700 1 --2- 172.21.15.32:0/2677708289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7530101270 0x7f753010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.934+0000 7f75365ff700 1 --2- 172.21.15.32:0/2677708289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301068c0 0x7f7530106ca0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.934+0000 7f75365ff700 1 --2- 172.21.15.32:0/2677708289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75301008b0 0x7f7530100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.935+0000 7f75365ff700 1 -- 172.21.15.32:0/2677708289 >> 172.21.15.32:0/2677708289 conn(0x7f75300fc380 msgr2=0x7f75300fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.935+0000 7f75365ff700 1 -- 172.21.15.32:0/2677708289 shutdown_connections 2023-08-11T15:54:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.935+0000 7f75365ff700 1 -- 172.21.15.32:0/2677708289 wait complete. 2023-08-11T15:54:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 Processor -- start 2023-08-11T15:54:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 -- start start 2023-08-11T15:54:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301008b0 0x7f753019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7530101270 0x7f753019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75301068c0 0x7f7530198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530199230 con 0x7f75301068c0 2023-08-11T15:54:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75301993a0 con 0x7f75301008b0 2023-08-11T15:54:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.936+0000 7f75365ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530199560 con 0x7f7530101270 2023-08-11T15:54:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301008b0 0x7f753019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301008b0 0x7f753019ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:34176/0 (socket says 172.21.15.32:34176) 2023-08-11T15:54:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 -- 172.21.15.32:0/4172443705 learned_addr learned my addr 172.21.15.32:0/4172443705 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f7534b9c700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75301068c0 0x7f7530198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752f7fe700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7530101270 0x7f753019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 -- 172.21.15.32:0/4172443705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7530101270 msgr2=0x7f753019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7530101270 0x7f753019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 -- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75301068c0 msgr2=0x7f7530198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75301068c0 0x7f7530198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 -- 172.21.15.32:0/4172443705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009520 con 0x7f75301008b0 2023-08-11T15:54:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f7534b9c700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75301068c0 0x7f7530198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:54:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.937+0000 7f752ffff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301008b0 0x7f753019ec90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7520005910 tx=0x7f7520004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.938+0000 7f752d7fa700 1 -- 172.21.15.32:0/4172443705 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752001d070 con 0x7f75301008b0 2023-08-11T15:54:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.938+0000 7f752d7fa700 1 -- 172.21.15.32:0/4172443705 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7520022470 con 0x7f75301008b0 2023-08-11T15:54:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.938+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75200095f0 con 0x7f75301008b0 2023-08-11T15:54:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.938+0000 7f752d7fa700 1 -- 172.21.15.32:0/4172443705 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520017650 con 0x7f75301008b0 2023-08-11T15:54:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.938+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75301998f0 con 0x7f75301008b0 2023-08-11T15:54:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.941+0000 7f752d7fa700 1 -- 172.21.15.32:0/4172443705 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f75200225e0 con 0x7f75301008b0 2023-08-11T15:54:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.942+0000 7f7549eacb80 1 Processor -- start 2023-08-11T15:54:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7549eacb80 1 -- start start 2023-08-11T15:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f752d7fa700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f75180a0140 0x7f75180a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd30e80 0x55e9cbd31260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2b630 0x55e9cbd2bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd37c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7549eacb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbc845d0 con 0x55e9cbd30e80 2023-08-11T15:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7549eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbc81ef0 con 0x55e9cbd2bff0 2023-08-11T15:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7549eacb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbc7d7e0 con 0x55e9cbd2b630 2023-08-11T15:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f752d7fa700 1 -- 172.21.15.32:0/4172443705 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f75200ccb90 con 0x7f75301008b0 2023-08-11T15:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f752f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2b630 0x55e9cbd2bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f752f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2b630 0x55e9cbd2bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:59456/0 (socket says 172.21.15.32:59456) 2023-08-11T15:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f752f7fe700 1 -- 172.21.15.32:0/199834995 learned_addr learned my addr 172.21.15.32:0/199834995 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f752ffff700 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd30e80 0x55e9cbd31260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.943+0000 7f7534b9c700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f75180a0140 0x7f75180a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f7534b9c700 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd37c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f752f7fe700 1 -- 172.21.15.32:0/199834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 msgr2=0x55e9cbd37c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f752f7fe700 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd37c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f752f7fe700 1 -- 172.21.15.32:0/199834995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd30e80 msgr2=0x55e9cbd31260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f752f7fe700 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd30e80 0x55e9cbd31260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f752f7fe700 1 -- 172.21.15.32:0/199834995 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9cbafd5d0 con 0x55e9cbd2b630 2023-08-11T15:54:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f752f7fe700 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2b630 0x55e9cbd2bab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e80 tx=0x7f7524011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad57e42a15335f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f7534b9c700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f75180a0140 0x7f75180a2600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55e9cbd26a30 tx=0x7f7528009920 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f75167fc700 1 -- 172.21.15.32:0/199834995 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524004250 con 0x55e9cbd2b630 2023-08-11T15:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f75167fc700 1 -- 172.21.15.32:0/199834995 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75240043b0 con 0x55e9cbd2b630 2023-08-11T15:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.944+0000 7f75167fc700 1 -- 172.21.15.32:0/199834995 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524019920 con 0x55e9cbd2b630 2023-08-11T15:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 -- 172.21.15.32:0/199834995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2b630 msgr2=0x55e9cbd2bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2b630 0x55e9cbd2bab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e80 tx=0x7f7524011720 comp rx=0 tx=0).stop 2023-08-11T15:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 -- 172.21.15.32:0/199834995 shutdown_connections 2023-08-11T15:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd30e80 0x55e9cbd31260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd37c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 --2- 172.21.15.32:0/199834995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2b630 0x55e9cbd2bab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 -- 172.21.15.32:0/199834995 >> 172.21.15.32:0/199834995 conn(0x55e9cbc588f0 msgr2=0x55e9cbd25e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 -- 172.21.15.32:0/199834995 shutdown_connections 2023-08-11T15:54:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.945+0000 7f7549eacb80 1 -- 172.21.15.32:0/199834995 wait complete. 2023-08-11T15:54:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.946+0000 7f7549eacb80 1 -- start start 2023-08-11T15:54:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.946+0000 7f7549eacb80 1 Processor -- start 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd3af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd30e80 0x55e9cbd34d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7549eacb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbc1ba60 con 0x55e9cbd2b630 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7549eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbc1b760 con 0x55e9cbd30e80 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7549eacb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbaff8c0 con 0x55e9cbd2bff0 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7549eacb80 10 client.0 _ll_register_callbacks cb 0x55e9cbc1ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd30e80 0x55e9cbd34d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f752ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd30e80 0x55e9cbd34d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:34192/0 (socket says 172.21.15.32:34192) 2023-08-11T15:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 -- 172.21.15.32:0/1059956266 learned_addr learned my addr 172.21.15.32:0/1059956266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f752f7fe700 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd3af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 -- 172.21.15.32:0/1059956266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2bff0 msgr2=0x55e9cbd3af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd3af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 -- 172.21.15.32:0/1059956266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 msgr2=0x55e9cbd3aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f7534b9c700 1 -- 172.21.15.32:0/1059956266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7524009ab0 con 0x55e9cbd30e80 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.947+0000 7f752ffff700 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f7534b9c700 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd30e80 0x55e9cbd34d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7528017ce0 tx=0x7f7528004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80c069cdae010f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f750f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f750f7fe700 1 -- 172.21.15.32:0/1059956266 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800e040 con 0x55e9cbd30e80 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9cbd1e7e0 con 0x55e9cbd30e80 2023-08-11T15:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f750f7fe700 1 -- 172.21.15.32:0/1059956266 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f752801e040 con 0x55e9cbd30e80 2023-08-11T15:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f750f7fe700 1 -- 172.21.15.32:0/1059956266 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7528007a90 con 0x55e9cbd30e80 2023-08-11T15:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f7549eacb80 10 client.26012 fetch_fsmap learned FSMap version 31 2023-08-11T15:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.948+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e9cbaa9d80 con 0x55e9cbd30e80 2023-08-11T15:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f750f7fe700 1 -- 172.21.15.32:0/1059956266 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9cbd1e7e0 con 0x55e9cbd30e80 2023-08-11T15:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f750f7fe700 1 -- 172.21.15.32:0/1059956266 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7528022410 con 0x55e9cbd30e80 2023-08-11T15:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f7549eacb80 10 client.26012 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f7549eacb80 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f7549eacb80 20 client.26012 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f7549eacb80 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd44f30 0x55e9cbd47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f7549eacb80 4 client.26012 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9cbcfa480 con 0x55e9cbd44f30 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.949+0000 7f752ffff700 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd44f30 0x55e9cbd47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.950+0000 7f752ffff700 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd44f30 0x55e9cbd47350 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acfb655da58ce427 server_cookie=a54d2b5a45804cc3 in_seq=0 out_seq=0 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.950+0000 7f750f7fe700 10 client.26012 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.950+0000 7f750f7fe700 1 -- 172.21.15.32:0/1059956266 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e9cbcfa480 con 0x55e9cbd44f30 2023-08-11T15:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.950+0000 7f750f7fe700 10 client.26012 handle_command_reply: tid=0 2023-08-11T15:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.950+0000 7f7549eacb80 1 client.26012 shutdown 2023-08-11T15:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.950+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd30e80 msgr2=0x55e9cbd34d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.950+0000 7f7549eacb80 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd30e80 0x55e9cbd34d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7528017ce0 tx=0x7f7528004b40 comp rx=0 tx=0).stop 2023-08-11T15:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 shutdown_connections 2023-08-11T15:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd44f30 0x55e9cbd47350 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd30e80 0x55e9cbd34d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f750f7fe700 0 client.26012 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 --2- 172.21.15.32:0/1059956266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd3af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 >> 172.21.15.32:0/1059956266 conn(0x55e9cbc588f0 msgr2=0x55e9cbd255d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 shutdown_connections 2023-08-11T15:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 1 -- 172.21.15.32:0/1059956266 wait complete. 2023-08-11T15:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.951+0000 7f7549eacb80 20 client.26012 trim_cache size 0 max 16384 2023-08-11T15:54:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 Processor -- start 2023-08-11T15:54:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 -- start start 2023-08-11T15:54:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd25d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbd27db0 con 0x55e9cbd2b630 2023-08-11T15:54:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbd26840 con 0x55e9cbd2bff0 2023-08-11T15:54:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7549eacb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbd30a90 con 0x55e9cbd30e80 2023-08-11T15:54:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f752ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f752ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59524/0 (socket says 172.21.15.32:59524) 2023-08-11T15:54:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f7534b9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f752ffff700 1 -- 172.21.15.32:0/799644959 learned_addr learned my addr 172.21.15.32:0/799644959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.968+0000 7f752f7fe700 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd25d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f752ffff700 1 -- 172.21.15.32:0/799644959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 msgr2=0x55e9cbd345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f752ffff700 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f752ffff700 1 -- 172.21.15.32:0/799644959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 msgr2=0x55e9cbd25d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f752ffff700 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd25d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f752ffff700 1 -- 172.21.15.32:0/799644959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9cba92c00 con 0x55e9cbd2b630 2023-08-11T15:54:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f752ffff700 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3a580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7520024000 tx=0x7f7520000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd8647a47fa2650 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f7516ffd700 1 -- 172.21.15.32:0/799644959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752009a660 con 0x55e9cbd2b630 2023-08-11T15:54:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f7516ffd700 1 -- 172.21.15.32:0/799644959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f752006a860 con 0x55e9cbd2b630 2023-08-11T15:54:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.969+0000 7f7516ffd700 1 -- 172.21.15.32:0/799644959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520022900 con 0x55e9cbd2b630 2023-08-11T15:54:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 -- 172.21.15.32:0/799644959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 msgr2=0x55e9cbd3a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3a580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7520024000 tx=0x7f7520000bc0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 -- 172.21.15.32:0/799644959 shutdown_connections 2023-08-11T15:54:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd3a580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 --2- 172.21.15.32:0/799644959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 -- 172.21.15.32:0/799644959 >> 172.21.15.32:0/799644959 conn(0x55e9cbc588f0 msgr2=0x55e9cbd2fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 -- 172.21.15.32:0/799644959 shutdown_connections 2023-08-11T15:54:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 -- 172.21.15.32:0/799644959 wait complete. 2023-08-11T15:54:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.970+0000 7f7549eacb80 1 -- start start 2023-08-11T15:54:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 1 Processor -- start 2023-08-11T15:54:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd37ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd38420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbd38d00 con 0x55e9cbd2b630 2023-08-11T15:54:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbd98120 con 0x55e9cbd2bff0 2023-08-11T15:54:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9cbd98260 con 0x55e9cbd30e80 2023-08-11T15:54:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f7549eacb80 10 client.0 _ll_register_callbacks cb 0x7f75280176b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.971+0000 7f752ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd37ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd37ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59530/0 (socket says 172.21.15.32:59530) 2023-08-11T15:54:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 -- 172.21.15.32:0/233651643 learned_addr learned my addr 172.21.15.32:0/233651643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f7534b9c700 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752f7fe700 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd38420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 -- 172.21.15.32:0/233651643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 msgr2=0x55e9cbd97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd97be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 -- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 msgr2=0x55e9cbd38420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd38420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 -- 172.21.15.32:0/233651643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75301998f0 con 0x55e9cbd2b630 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f752ffff700 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd37ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7520099c60 tx=0x7f7520022c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860b11cce951614c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f750f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f750f7fe700 1 -- 172.21.15.32:0/233651643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752009a660 con 0x55e9cbd2b630 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f750f7fe700 1 -- 172.21.15.32:0/233651643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f75200a1ce0 con 0x55e9cbd2b630 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.972+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75240043b0 con 0x55e9cbd2b630 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f750f7fe700 1 -- 172.21.15.32:0/233651643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f752009dd90 con 0x55e9cbd2b630 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 10 client.15996 fetch_fsmap learned FSMap version 31 2023-08-11T15:54:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e9cbad01f0 con 0x55e9cbd2b630 2023-08-11T15:54:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f750f7fe700 1 -- 172.21.15.32:0/233651643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75240043b0 con 0x55e9cbd2b630 2023-08-11T15:54:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f750f7fe700 1 -- 172.21.15.32:0/233651643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7520099450 con 0x55e9cbd2b630 2023-08-11T15:54:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T15:54:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 20 client.15996 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd994f0 0x55e9cbd9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 4 client.15996 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:54:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e9cbc767c0 con 0x55e9cbd994f0 2023-08-11T15:54:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.973+0000 7f752f7fe700 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd994f0 0x55e9cbd9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.974+0000 7f752f7fe700 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd994f0 0x55e9cbd9b910 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87724fbc25da96d2 server_cookie=74a5febe29589cd1 in_seq=0 out_seq=0 2023-08-11T15:54:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:21.974+0000 7f750f7fe700 10 client.15996 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.314+0000 7f750f7fe700 1 -- 172.21.15.32:0/233651643 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e9cbc767c0 con 0x55e9cbd994f0 2023-08-11T15:54:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.314+0000 7f750f7fe700 10 client.15996 handle_command_reply: tid=0 2023-08-11T15:54:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.314+0000 7f7549eacb80 1 client.15996 shutdown 2023-08-11T15:54:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 msgr2=0x55e9cbd37ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd37ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7520099c60 tx=0x7f7520022c40 comp rx=0 tx=0).stop 2023-08-11T15:54:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 shutdown_connections 2023-08-11T15:54:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x55e9cbd994f0 0x55e9cbd9b910 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e9cbd2b630 0x55e9cbd37ee0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9cbd30e80 0x55e9cbd97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 --2- 172.21.15.32:0/233651643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9cbd2bff0 0x55e9cbd38420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f750f7fe700 0 client.15996 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 >> 172.21.15.32:0/233651643 conn(0x55e9cbc588f0 msgr2=0x55e9cbd2a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 shutdown_connections 2023-08-11T15:54:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.315+0000 7f7549eacb80 1 -- 172.21.15.32:0/233651643 wait complete. 2023-08-11T15:54:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.316+0000 7f7549eacb80 20 client.15996 trim_cache size 0 max 16384 2023-08-11T15:54:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:54:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:54:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:54:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:54:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.316+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f75180a0140 msgr2=0x7f75180a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.316+0000 7f75365ff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f75180a0140 0x7f75180a2600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55e9cbd26a30 tx=0x7f7528009920 comp rx=0 tx=0).stop 2023-08-11T15:54:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.316+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301008b0 msgr2=0x7f753019ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.316+0000 7f75365ff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301008b0 0x7f753019ec90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7520005910 tx=0x7f7520004710 comp rx=0 tx=0).stop 2023-08-11T15:54:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 shutdown_connections 2023-08-11T15:54:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f75180a0140 0x7f75180a2600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75301068c0 0x7f7530198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75301008b0 0x7f753019ec90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 --2- 172.21.15.32:0/4172443705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7530101270 0x7f753019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 >> 172.21.15.32:0/4172443705 conn(0x7f75300fc380 msgr2=0x7f753010bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 shutdown_connections 2023-08-11T15:54:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:22.317+0000 7f75365ff700 1 -- 172.21.15.32:0/4172443705 wait complete. 2023-08-11T15:54:22.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:22 smithi134 ceph-mon[130447]: pgmap v242: 129 pgs: 129 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-08-11T15:54:22.721 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:54:22 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: ignoring --setuser ceph since I am not root 2023-08-11T15:54:22.722 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:54:22 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: ignoring --setgroup ceph since I am not root 2023-08-11T15:54:22.722 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:54:22 smithi134 ceph-mds[155206]: -- 172.21.15.134:0/1029995685 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5617c7ad51e0 con 0x5617c7aaac00 2023-08-11T15:54:22.722 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:54:22 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: starting mds.h at 2023-08-11T15:54:22.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 862121 11.03 MB/sec execute 278 sec latency 129.618 ms 2023-08-11T15:54:22.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:22 smithi032 ceph-mon[97103]: pgmap v242: 129 pgs: 129 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-08-11T15:54:22.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:22 smithi192 ceph-mon[130806]: pgmap v242: 129 pgs: 129 active+clean; 76 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-08-11T15:54:23.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:23 smithi134 ceph-mon[130447]: daemon mds.h restarted 2023-08-11T15:54:23.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 863803 11.02 MB/sec execute 279 sec latency 19.708 ms 2023-08-11T15:54:23.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:23 smithi032 ceph-mon[97103]: daemon mds.h restarted 2023-08-11T15:54:23.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:23 smithi192 ceph-mon[130806]: daemon mds.h restarted 2023-08-11T15:54:24.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:24 smithi134 ceph-mon[130447]: pgmap v243: 129 pgs: 129 active+clean; 78 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 368 op/s 2023-08-11T15:54:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:24 smithi134 ceph-mon[130447]: osdmap e103: 12 total, 12 up, 12 in 2023-08-11T15:54:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:24 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6836/4011211123,v1:172.21.15.134:6837/4011211123] up:boot 2023-08-11T15:54:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:24 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:54:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:24 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:54:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:24 smithi134 ceph-mon[130447]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:54:24.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 866521 11.04 MB/sec execute 280 sec latency 27.945 ms 2023-08-11T15:54:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:24 smithi032 ceph-mon[97103]: pgmap v243: 129 pgs: 129 active+clean; 78 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 368 op/s 2023-08-11T15:54:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:24 smithi032 ceph-mon[97103]: osdmap e103: 12 total, 12 up, 12 in 2023-08-11T15:54:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:24 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6836/4011211123,v1:172.21.15.134:6837/4011211123] up:boot 2023-08-11T15:54:24.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:24 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:54:24.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:24 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:54:24.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:24 smithi032 ceph-mon[97103]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:54:24.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:24 smithi192 ceph-mon[130806]: pgmap v243: 129 pgs: 129 active+clean; 78 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 368 op/s 2023-08-11T15:54:24.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:24 smithi192 ceph-mon[130806]: osdmap e103: 12 total, 12 up, 12 in 2023-08-11T15:54:24.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:24 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6836/4011211123,v1:172.21.15.134:6837/4011211123] up:boot 2023-08-11T15:54:24.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:24 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T15:54:24.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:24 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:54:24.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:24 smithi192 ceph-mon[130806]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-08-11T15:54:25.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 867781 11.02 MB/sec execute 281 sec latency 66.716 ms 2023-08-11T15:54:26.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:26 smithi134 ceph-mon[130447]: pgmap v245: 129 pgs: 129 active+clean; 78 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 446 op/s 2023-08-11T15:54:26.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 870145 11.03 MB/sec execute 282 sec latency 55.729 ms 2023-08-11T15:54:26.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:26 smithi032 ceph-mon[97103]: pgmap v245: 129 pgs: 129 active+clean; 78 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 446 op/s 2023-08-11T15:54:26.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:26 smithi192 ceph-mon[130806]: pgmap v245: 129 pgs: 129 active+clean; 78 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 446 op/s 2023-08-11T15:54:27.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 872235 11.02 MB/sec execute 283 sec latency 16.526 ms 2023-08-11T15:54:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:28 smithi032 ceph-mon[97103]: pgmap v246: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 438 op/s 2023-08-11T15:54:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:28 smithi192 ceph-mon[130806]: pgmap v246: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 438 op/s 2023-08-11T15:54:28.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:28 smithi134 ceph-mon[130447]: pgmap v246: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 438 op/s 2023-08-11T15:54:28.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 874187 11.03 MB/sec execute 284 sec latency 39.830 ms 2023-08-11T15:54:29.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 876594 11.03 MB/sec execute 285 sec latency 21.745 ms 2023-08-11T15:54:30.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:30 smithi134 ceph-mon[130447]: pgmap v247: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 401 op/s 2023-08-11T15:54:30.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 877908 11.02 MB/sec execute 286 sec latency 93.640 ms 2023-08-11T15:54:30.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:30 smithi032 ceph-mon[97103]: pgmap v247: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 401 op/s 2023-08-11T15:54:30.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:30 smithi192 ceph-mon[130806]: pgmap v247: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 401 op/s 2023-08-11T15:54:31.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 880111 11.02 MB/sec execute 287 sec latency 52.106 ms 2023-08-11T15:54:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:32 smithi134 ceph-mon[130447]: pgmap v248: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 401 op/s 2023-08-11T15:54:32.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 881929 11.02 MB/sec execute 288 sec latency 18.863 ms 2023-08-11T15:54:32.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:32 smithi032 ceph-mon[97103]: pgmap v248: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 401 op/s 2023-08-11T15:54:32.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:32 smithi192 ceph-mon[130806]: pgmap v248: 129 pgs: 129 active+clean; 69 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 401 op/s 2023-08-11T15:54:33.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 884346 11.03 MB/sec execute 289 sec latency 11.805 ms 2023-08-11T15:54:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:34 smithi134 ceph-mon[130447]: pgmap v249: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 276 op/s 2023-08-11T15:54:34.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 886218 11.01 MB/sec execute 290 sec latency 19.314 ms 2023-08-11T15:54:34.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:34 smithi032 ceph-mon[97103]: pgmap v249: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 276 op/s 2023-08-11T15:54:34.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:34 smithi192 ceph-mon[130806]: pgmap v249: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 276 op/s 2023-08-11T15:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.472+0000 7fa7161ff700 1 -- 172.21.15.32:0/3448987389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101068f0 msgr2=0x7fa710106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.472+0000 7fa7161ff700 1 --2- 172.21.15.32:0/3448987389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101068f0 0x7fa710106cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa700009990 tx=0x7fa700009ca0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.473+0000 7fa7161ff700 1 -- 172.21.15.32:0/3448987389 shutdown_connections 2023-08-11T15:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.473+0000 7fa7161ff700 1 --2- 172.21.15.32:0/3448987389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7101012a0 0x7fa71010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.473+0000 7fa7161ff700 1 --2- 172.21.15.32:0/3448987389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101068f0 0x7fa710106cd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.473+0000 7fa7161ff700 1 --2- 172.21.15.32:0/3448987389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101008e0 0x7fa710100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.473+0000 7fa7161ff700 1 -- 172.21.15.32:0/3448987389 >> 172.21.15.32:0/3448987389 conn(0x7fa7100fc3f0 msgr2=0x7fa7100fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.473+0000 7fa7161ff700 1 -- 172.21.15.32:0/3448987389 shutdown_connections 2023-08-11T15:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.473+0000 7fa7161ff700 1 -- 172.21.15.32:0/3448987389 wait complete. 2023-08-11T15:54:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 Processor -- start 2023-08-11T15:54:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 -- start start 2023-08-11T15:54:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7101008e0 0x7fa71019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101012a0 0x7fa71019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101068f0 0x7fa710198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7101991f0 con 0x7fa7101008e0 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710199360 con 0x7fa7101012a0 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.474+0000 7fa7161ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710199520 con 0x7fa7101068f0 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101012a0 0x7fa71019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101068f0 0x7fa710198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7101008e0 0x7fa71019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101068f0 0x7fa710198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:41426/0 (socket says 172.21.15.32:41426) 2023-08-11T15:54:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101012a0 0x7fa71019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:59840/0 (socket says 172.21.15.32:59840) 2023-08-11T15:54:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70ffff700 1 -- 172.21.15.32:0/529356934 learned_addr learned my addr 172.21.15.32:0/529356934 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 -- 172.21.15.32:0/529356934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101068f0 msgr2=0x7fa710198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101068f0 0x7fa710198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 -- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7101008e0 msgr2=0x7fa71019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7101008e0 0x7fa71019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 -- 172.21.15.32:0/529356934 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000095f0 con 0x7fa7101012a0 2023-08-11T15:54:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70ffff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101068f0 0x7fa710198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:54:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70effd700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101012a0 0x7fa71019f190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa70400e9f0 tx=0x7fa70400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.475+0000 7fa70cff9700 1 -- 172.21.15.32:0/529356934 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70400c9c0 con 0x7fa7101012a0 2023-08-11T15:54:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.476+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa710199800 con 0x7fa7101012a0 2023-08-11T15:54:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.476+0000 7fa70cff9700 1 -- 172.21.15.32:0/529356934 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa704010430 con 0x7fa7101012a0 2023-08-11T15:54:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.476+0000 7fa70cff9700 1 -- 172.21.15.32:0/529356934 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704018680 con 0x7fa7101012a0 2023-08-11T15:54:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.476+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101a5c20 con 0x7fa7101012a0 2023-08-11T15:54:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.479+0000 7fa70cff9700 1 -- 172.21.15.32:0/529356934 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa704010860 con 0x7fa7101012a0 2023-08-11T15:54:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.480+0000 7fa729acbb80 1 Processor -- start 2023-08-11T15:54:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa729acbb80 1 -- start start 2023-08-11T15:54:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b18260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b12ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa729acbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64a6b5d0 con 0x562e64b12ff0 2023-08-11T15:54:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa729acbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64a68ef0 con 0x562e64b12630 2023-08-11T15:54:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa729acbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64a647e0 con 0x562e64b17e80 2023-08-11T15:54:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa70cff9700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa6f80a0140 0x7fa6f80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b12ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa70effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b12ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:59846/0 (socket says 172.21.15.32:59846) 2023-08-11T15:54:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.481+0000 7fa70f7fe700 1 -- 172.21.15.32:0/3239228481 learned_addr learned my addr 172.21.15.32:0/3239228481 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70ffff700 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b18260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70cff9700 1 -- 172.21.15.32:0/529356934 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa704014070 con 0x7fa7101012a0 2023-08-11T15:54:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70ffff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa6f80a0140 0x7fa6f80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70f7fe700 1 -- 172.21.15.32:0/3239228481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 msgr2=0x562e64b18260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b18260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70f7fe700 1 -- 172.21.15.32:0/3239228481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 msgr2=0x562e64b1ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70f7fe700 1 -- 172.21.15.32:0/3239228481 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e648e45d0 con 0x562e64b12630 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b12ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa700005ea0 tx=0x7fa700005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a3055550da4ef26 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa70ffff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa6f80a0140 0x7fa6f80a2600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa708009d30 tx=0x7fa70800b040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.482+0000 7fa6f5ffb700 1 -- 172.21.15.32:0/3239228481 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70001d070 con 0x562e64b12630 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa6f5ffb700 1 -- 172.21.15.32:0/3239228481 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70001e040 con 0x562e64b12630 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa6f5ffb700 1 -- 172.21.15.32:0/3239228481 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7000047f0 con 0x562e64b12630 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa729acbb80 1 -- 172.21.15.32:0/3239228481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 msgr2=0x562e64b12ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa729acbb80 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b12ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa700005ea0 tx=0x7fa700005ed0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa729acbb80 1 -- 172.21.15.32:0/3239228481 shutdown_connections 2023-08-11T15:54:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa729acbb80 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa729acbb80 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b18260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa729acbb80 1 --2- 172.21.15.32:0/3239228481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b12ab0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.483+0000 7fa729acbb80 1 -- 172.21.15.32:0/3239228481 >> 172.21.15.32:0/3239228481 conn(0x562e64a3f8f0 msgr2=0x562e64b0ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.484+0000 7fa729acbb80 1 -- 172.21.15.32:0/3239228481 shutdown_connections 2023-08-11T15:54:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.484+0000 7fa729acbb80 1 -- 172.21.15.32:0/3239228481 wait complete. 2023-08-11T15:54:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.484+0000 7fa729acbb80 1 -- start start 2023-08-11T15:54:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.484+0000 7fa729acbb80 1 Processor -- start 2023-08-11T15:54:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b218c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b12ff0 0x562e64b21e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b17e80 0x562e64b1bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa729acbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64a02a60 con 0x562e64b17e80 2023-08-11T15:54:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa729acbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64a02760 con 0x562e64b12630 2023-08-11T15:54:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa729acbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e648e68c0 con 0x562e64b12ff0 2023-08-11T15:54:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa729acbb80 10 client.0 _ll_register_callbacks cb 0x562e64a05a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b218c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:59850/0 (socket says 172.21.15.32:59850) 2023-08-11T15:54:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa70f7fe700 1 -- 172.21.15.32:0/1584143988 learned_addr learned my addr 172.21.15.32:0/1584143988 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa70effd700 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b12ff0 0x562e64b21e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa70ffff700 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b17e80 0x562e64b1bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa70f7fe700 1 -- 172.21.15.32:0/1584143988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b12ff0 msgr2=0x562e64b21e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.485+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b12ff0 0x562e64b21e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa70f7fe700 1 -- 172.21.15.32:0/1584143988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b17e80 msgr2=0x562e64b1bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b17e80 0x562e64b1bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa70f7fe700 1 -- 172.21.15.32:0/1584143988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71010e0b0 con 0x562e64b12630 2023-08-11T15:54:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b218c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa700000c00 tx=0x7fa70000fbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3d4d1e1e3205a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa6eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:54:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa6eeffd700 1 -- 172.21.15.32:0/1584143988 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70001d040 con 0x562e64b12630 2023-08-11T15:54:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e64b057e0 con 0x562e64b12630 2023-08-11T15:54:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa6eeffd700 1 -- 172.21.15.32:0/1584143988 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa700005980 con 0x562e64b12630 2023-08-11T15:54:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa6eeffd700 1 -- 172.21.15.32:0/1584143988 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa700005bc0 con 0x562e64b12630 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa729acbb80 10 client.26036 fetch_fsmap learned FSMap version 33 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.486+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562e64890d80 con 0x562e64b12630 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa6eeffd700 1 -- 172.21.15.32:0/1584143988 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70003b020 con 0x562e64b12630 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa6eeffd700 1 -- 172.21.15.32:0/1584143988 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa700004bf0 con 0x562e64b12630 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa729acbb80 10 client.26036 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa729acbb80 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa729acbb80 20 client.26036 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa729acbb80 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b2b920 0x562e64b2dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa729acbb80 4 client.26036 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:54:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e64ae1480 con 0x562e64b2b920 2023-08-11T15:54:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.487+0000 7fa70effd700 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b2b920 0x562e64b2dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.488+0000 7fa70effd700 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b2b920 0x562e64b2dd10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3742c4bfa3fbdb7c server_cookie=8864f75a97c5271c in_seq=0 out_seq=0 2023-08-11T15:54:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.488+0000 7fa6eeffd700 10 client.26036 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.488+0000 7fa6eeffd700 1 -- 172.21.15.32:0/1584143988 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562e64ae1480 con 0x562e64b2b920 2023-08-11T15:54:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.488+0000 7fa6eeffd700 10 client.26036 handle_command_reply: tid=0 2023-08-11T15:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.488+0000 7fa729acbb80 1 client.26036 shutdown 2023-08-11T15:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.488+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 msgr2=0x562e64b218c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.488+0000 7fa729acbb80 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b218c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa700000c00 tx=0x7fa70000fbf0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 shutdown_connections 2023-08-11T15:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b2b920 0x562e64b2dd10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b17e80 0x562e64b1bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b218c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 --2- 172.21.15.32:0/1584143988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b12ff0 0x562e64b21e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa6eeffd700 0 client.26036 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 >> 172.21.15.32:0/1584143988 conn(0x562e64a3f8f0 msgr2=0x562e64b0c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 shutdown_connections 2023-08-11T15:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 1 -- 172.21.15.32:0/1584143988 wait complete. 2023-08-11T15:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.489+0000 7fa729acbb80 20 client.26036 trim_cache size 0 max 16384 2023-08-11T15:54:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.504+0000 7fa729acbb80 1 Processor -- start 2023-08-11T15:54:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa729acbb80 1 -- start start 2023-08-11T15:54:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12630 0x562e64b12a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12ff0 0x562e64b0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b14860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa729acbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64b0d290 con 0x562e64b12630 2023-08-11T15:54:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa729acbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64b0d3d0 con 0x562e64b12ff0 2023-08-11T15:54:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa729acbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64b16500 con 0x562e64b17e80 2023-08-11T15:54:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12ff0 0x562e64b0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12630 0x562e64b12a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12ff0 0x562e64b0cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:59864/0 (socket says 172.21.15.32:59864) 2023-08-11T15:54:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70effd700 1 -- 172.21.15.32:0/261960844 learned_addr learned my addr 172.21.15.32:0/261960844 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70ffff700 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b14860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70f7fe700 1 -- 172.21.15.32:0/261960844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 msgr2=0x562e64b14860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b14860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70f7fe700 1 -- 172.21.15.32:0/261960844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12ff0 msgr2=0x562e64b0cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.505+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12ff0 0x562e64b0cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.506+0000 7fa70f7fe700 1 -- 172.21.15.32:0/261960844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e64879c00 con 0x562e64b12630 2023-08-11T15:54:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.506+0000 7fa70effd700 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12ff0 0x562e64b0cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:54:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.506+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12630 0x562e64b12a10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7000098e0 tx=0x7fa700009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad09f1381b7607b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.506+0000 7fa6f67fc700 1 -- 172.21.15.32:0/261960844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70000bb30 con 0x562e64b12630 2023-08-11T15:54:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.506+0000 7fa6f67fc700 1 -- 172.21.15.32:0/261960844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70003b030 con 0x562e64b12630 2023-08-11T15:54:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.506+0000 7fa6f67fc700 1 -- 172.21.15.32:0/261960844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700005560 con 0x562e64b12630 2023-08-11T15:54:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 -- 172.21.15.32:0/261960844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12630 msgr2=0x562e64b12a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12630 0x562e64b12a10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7000098e0 tx=0x7fa700009310 comp rx=0 tx=0).stop 2023-08-11T15:54:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 -- 172.21.15.32:0/261960844 shutdown_connections 2023-08-11T15:54:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12630 0x562e64b12a10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b14860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 --2- 172.21.15.32:0/261960844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12ff0 0x562e64b0cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 -- 172.21.15.32:0/261960844 >> 172.21.15.32:0/261960844 conn(0x562e64a3f8f0 msgr2=0x562e64b15df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 -- 172.21.15.32:0/261960844 shutdown_connections 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 -- 172.21.15.32:0/261960844 wait complete. 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 -- start start 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.507+0000 7fa729acbb80 1 Processor -- start 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b1f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa729acbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b7e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa729acbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64b7eb00 con 0x562e64b12ff0 2023-08-11T15:54:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa729acbb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64b7ec40 con 0x562e64b12630 2023-08-11T15:54:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa729acbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e64b7edb0 con 0x562e64b17e80 2023-08-11T15:54:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa729acbb80 10 client.0 _ll_register_callbacks cb 0x7fa704010b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b1f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa70effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa70f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b1f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:59874/0 (socket says 172.21.15.32:59874) 2023-08-11T15:54:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa70f7fe700 1 -- 172.21.15.32:0/2909042186 learned_addr learned my addr 172.21.15.32:0/2909042186 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.508+0000 7fa70ffff700 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b7e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa70f7fe700 1 -- 172.21.15.32:0/2909042186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 msgr2=0x562e64b7e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b7e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa70f7fe700 1 -- 172.21.15.32:0/2909042186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 msgr2=0x562e64b1f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa70f7fe700 1 -- 172.21.15.32:0/2909042186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101a5c20 con 0x562e64b12630 2023-08-11T15:54:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa70effd700 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:54:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa70f7fe700 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b1f0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa7000224f0 tx=0x7fa700005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae5cd241e69c6f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa6eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-08-11T15:54:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa6eeffd700 1 -- 172.21.15.32:0/2909042186 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70001d030 con 0x562e64b12630 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa70001e040 con 0x562e64b12630 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa6eeffd700 1 -- 172.21.15.32:0/2909042186 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fa70003c080 con 0x562e64b12630 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa6eeffd700 1 -- 172.21.15.32:0/2909042186 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa700022d90 con 0x562e64b12630 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa729acbb80 10 client.26042 fetch_fsmap learned FSMap version 33 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.509+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562e64890d80 con 0x562e64b12630 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa6eeffd700 1 -- 172.21.15.32:0/2909042186 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70001e040 con 0x562e64b12630 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa6eeffd700 1 -- 172.21.15.32:0/2909042186 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa700004950 con 0x562e64b12630 2023-08-11T15:54:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa729acbb80 10 client.26042 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T15:54:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa729acbb80 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa729acbb80 20 client.26042 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa729acbb80 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b80090 0x562e64b82480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa729acbb80 4 client.26042 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:54:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e64a5d7c0 con 0x562e64b80090 2023-08-11T15:54:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.510+0000 7fa70effd700 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b80090 0x562e64b82480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.511+0000 7fa70effd700 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b80090 0x562e64b82480 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33ebb3fc9f6537e server_cookie=2f0a48bc5c322f8a in_seq=0 out_seq=0 2023-08-11T15:54:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.511+0000 7fa6eeffd700 10 client.26042 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:35.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 887885 11.02 MB/sec execute 291 sec latency 174.226 ms 2023-08-11T15:54:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa6eeffd700 1 -- 172.21.15.32:0/2909042186 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e64a5d7c0 con 0x562e64b80090 2023-08-11T15:54:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa6eeffd700 10 client.26042 handle_command_reply: tid=0 2023-08-11T15:54:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 client.26042 shutdown 2023-08-11T15:54:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 msgr2=0x562e64b1f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b1f0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa7000224f0 tx=0x7fa700005a50 comp rx=0 tx=0).stop 2023-08-11T15:54:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 shutdown_connections 2023-08-11T15:54:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x562e64b80090 0x562e64b82480 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e64b17e80 0x562e64b7e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e64b12630 0x562e64b1f0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 --2- 172.21.15.32:0/2909042186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562e64b12ff0 0x562e64b1f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa6eeffd700 0 client.26042 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.891+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 >> 172.21.15.32:0/2909042186 conn(0x562e64a3f8f0 msgr2=0x562e64b11620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.892+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 shutdown_connections 2023-08-11T15:54:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.892+0000 7fa729acbb80 1 -- 172.21.15.32:0/2909042186 wait complete. 2023-08-11T15:54:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.893+0000 7fa729acbb80 20 client.26042 trim_cache size 0 max 16384 2023-08-11T15:54:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:54:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:54:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:54:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:54:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa6f80a0140 msgr2=0x7fa6f80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa6f80a0140 0x7fa6f80a2600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa708009d30 tx=0x7fa70800b040 comp rx=0 tx=0).stop 2023-08-11T15:54:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101012a0 msgr2=0x7fa71019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101012a0 0x7fa71019f190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa70400e9f0 tx=0x7fa70400ed00 comp rx=0 tx=0).stop 2023-08-11T15:54:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 shutdown_connections 2023-08-11T15:54:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa6f80a0140 0x7fa6f80a2600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7101008e0 0x7fa71019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7101068f0 0x7fa710198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 --2- 172.21.15.32:0/529356934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7101012a0 0x7fa71019f190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 >> 172.21.15.32:0/529356934 conn(0x7fa7100fc3f0 msgr2=0x7fa71010bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 shutdown_connections 2023-08-11T15:54:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:35.894+0000 7fa7161ff700 1 -- 172.21.15.32:0/529356934 wait complete. 2023-08-11T15:54:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:36 smithi134 ceph-mon[130447]: pgmap v250: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-08-11T15:54:36.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 889175 11.00 MB/sec execute 292 sec latency 199.831 ms 2023-08-11T15:54:36.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:36 smithi032 ceph-mon[97103]: pgmap v250: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-08-11T15:54:36.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:36 smithi192 ceph-mon[130806]: pgmap v250: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-08-11T15:54:37.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 891491 11.01 MB/sec execute 293 sec latency 56.748 ms 2023-08-11T15:54:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:38 smithi032 ceph-mon[97103]: pgmap v251: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2023-08-11T15:54:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:38 smithi192 ceph-mon[130806]: pgmap v251: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2023-08-11T15:54:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:38 smithi134 ceph-mon[130447]: pgmap v251: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2023-08-11T15:54:38.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 893539 11.00 MB/sec execute 294 sec latency 16.700 ms 2023-08-11T15:54:39.787 INFO:tasks.workunit.client.0.smithi032.stdout: 1 895394 11.00 MB/sec execute 295 sec latency 21.002 ms 2023-08-11T15:54:40.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:40 smithi134 ceph-mon[130447]: pgmap v252: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2023-08-11T15:54:40.788 INFO:tasks.workunit.client.0.smithi032.stdout: 1 897417 11.00 MB/sec execute 296 sec latency 61.654 ms 2023-08-11T15:54:40.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:40 smithi032 ceph-mon[97103]: pgmap v252: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2023-08-11T15:54:40.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:40 smithi192 ceph-mon[130806]: pgmap v252: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2023-08-11T15:54:41.788 INFO:tasks.workunit.client.0.smithi032.stdout: 1 899026 10.99 MB/sec execute 297 sec latency 83.029 ms 2023-08-11T15:54:42.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:42 smithi134 ceph-mon[130447]: pgmap v253: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2023-08-11T15:54:42.789 INFO:tasks.workunit.client.0.smithi032.stdout: 1 901562 11.00 MB/sec execute 298 sec latency 32.932 ms 2023-08-11T15:54:42.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:42 smithi032 ceph-mon[97103]: pgmap v253: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2023-08-11T15:54:42.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:42 smithi192 ceph-mon[130806]: pgmap v253: 129 pgs: 129 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2023-08-11T15:54:43.789 INFO:tasks.workunit.client.0.smithi032.stdout: 1 903386 11.00 MB/sec execute 299 sec latency 13.146 ms 2023-08-11T15:54:44.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:44 smithi134 ceph-mon[130447]: pgmap v254: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2023-08-11T15:54:44.791 INFO:tasks.workunit.client.0.smithi032.stdout: 1 905673 11.01 MB/sec execute 300 sec latency 16.154 ms 2023-08-11T15:54:44.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:44 smithi032 ceph-mon[97103]: pgmap v254: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2023-08-11T15:54:44.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:44 smithi192 ceph-mon[130806]: pgmap v254: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2023-08-11T15:54:45.792 INFO:tasks.workunit.client.0.smithi032.stdout: 1 907550 10.99 MB/sec execute 301 sec latency 33.472 ms 2023-08-11T15:54:46.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:46 smithi134 ceph-mon[130447]: pgmap v255: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 289 op/s 2023-08-11T15:54:46.792 INFO:tasks.workunit.client.0.smithi032.stdout: 1 909432 11.00 MB/sec execute 302 sec latency 116.312 ms 2023-08-11T15:54:46.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:46 smithi032 ceph-mon[97103]: pgmap v255: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 289 op/s 2023-08-11T15:54:46.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:46 smithi192 ceph-mon[130806]: pgmap v255: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 289 op/s 2023-08-11T15:54:47.792 INFO:tasks.workunit.client.0.smithi032.stdout: 1 911207 10.99 MB/sec execute 303 sec latency 104.561 ms 2023-08-11T15:54:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:48 smithi032 ceph-mon[97103]: pgmap v256: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 373 op/s 2023-08-11T15:54:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:48 smithi192 ceph-mon[130806]: pgmap v256: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 373 op/s 2023-08-11T15:54:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:48 smithi134 ceph-mon[130447]: pgmap v256: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 373 op/s 2023-08-11T15:54:48.792 INFO:tasks.workunit.client.0.smithi032.stdout: 1 913092 10.99 MB/sec execute 304 sec latency 45.118 ms 2023-08-11T15:54:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 -- 172.21.15.32:0/3965073078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 msgr2=0x7fe970106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 --2- 172.21.15.32:0/3965073078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 0x7fe970106c90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009990 tx=0x7fe96c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 -- 172.21.15.32:0/3965073078 shutdown_connections 2023-08-11T15:54:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 --2- 172.21.15.32:0/3965073078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 0x7fe97010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 --2- 172.21.15.32:0/3965073078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 0x7fe970106c90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 --2- 172.21.15.32:0/3965073078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9701008a0 0x7fe970100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 -- 172.21.15.32:0/3965073078 >> 172.21.15.32:0/3965073078 conn(0x7fe9700fc370 msgr2=0x7fe9700fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 -- 172.21.15.32:0/3965073078 shutdown_connections 2023-08-11T15:54:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.060+0000 7fe977dff700 1 -- 172.21.15.32:0/3965073078 wait complete. 2023-08-11T15:54:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 Processor -- start 2023-08-11T15:54:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 -- start start 2023-08-11T15:54:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9701008a0 0x7fe970072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 0x7fe970073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 0x7fe9701a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701a50d0 con 0x7fe9701008a0 2023-08-11T15:54:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701a5210 con 0x7fe970101260 2023-08-11T15:54:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe977dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701a5380 con 0x7fe9701068b0 2023-08-11T15:54:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe97539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 0x7fe970073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe97539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 0x7fe970073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:58424/0 (socket says 172.21.15.32:58424) 2023-08-11T15:54:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.062+0000 7fe97539a700 1 -- 172.21.15.32:0/1206409612 learned_addr learned my addr 172.21.15.32:0/1206409612 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97639c700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 0x7fe9701a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe975b9b700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9701008a0 0x7fe970072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97639c700 1 -- 172.21.15.32:0/1206409612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 msgr2=0x7fe970073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97639c700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 0x7fe970073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97639c700 1 -- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9701008a0 msgr2=0x7fe970072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97639c700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9701008a0 0x7fe970072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97639c700 1 -- 172.21.15.32:0/1206409612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0095f0 con 0x7fe9701068b0 2023-08-11T15:54:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97539a700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 0x7fe970073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:54:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe97639c700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 0x7fe9701a4b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe96800efc0 tx=0x7fe96800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe966ffd700 1 -- 172.21.15.32:0/1206409612 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968018070 con 0x7fe9701068b0 2023-08-11T15:54:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.063+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9701a5600 con 0x7fe9701068b0 2023-08-11T15:54:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.064+0000 7fe966ffd700 1 -- 172.21.15.32:0/1206409612 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96800f040 con 0x7fe9701068b0 2023-08-11T15:54:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.064+0000 7fe966ffd700 1 -- 172.21.15.32:0/1206409612 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800cd40 con 0x7fe9701068b0 2023-08-11T15:54:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.064+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9701a5b10 con 0x7fe9701068b0 2023-08-11T15:54:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.067+0000 7fe966ffd700 1 -- 172.21.15.32:0/1206409612 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fe968003a00 con 0x7fe9701068b0 2023-08-11T15:54:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe966ffd700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe95c0a00c0 0x7fe95c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 Processor -- start 2023-08-11T15:54:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe966ffd700 1 -- 172.21.15.32:0/1206409612 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe9680cb550 con 0x7fe9701068b0 2023-08-11T15:54:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe975b9b700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe95c0a00c0 0x7fe95c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 -- start start 2023-08-11T15:54:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db30f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db3101c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db304e5d0 con 0x555db30fae80 2023-08-11T15:54:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db304bef0 con 0x555db30f5630 2023-08-11T15:54:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.069+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db30477e0 con 0x555db30f5ff0 2023-08-11T15:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db30f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db30f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:58428/0 (socket says 172.21.15.32:58428) 2023-08-11T15:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 -- 172.21.15.32:0/3028375270 learned_addr learned my addr 172.21.15.32:0/3028375270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe975b9b700 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97639c700 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db3101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 -- 172.21.15.32:0/3028375270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 msgr2=0x555db3101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db3101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 -- 172.21.15.32:0/3028375270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 msgr2=0x555db30fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30fb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 -- 172.21.15.32:0/3028375270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db2ec75d0 con 0x555db30f5630 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe975b9b700 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30fb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe975b9b700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe95c0a00c0 0x7fe95c0a2580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b2e0 tx=0x7fe96c00b850 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.070+0000 7fe97539a700 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db30f5ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe960011680 tx=0x7fe960011990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0409c71e2bf88d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.071+0000 7fe95bfff700 1 -- 172.21.15.32:0/3028375270 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960004030 con 0x555db30f5630 2023-08-11T15:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.071+0000 7fe95bfff700 1 -- 172.21.15.32:0/3028375270 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe960003680 con 0x555db30f5630 2023-08-11T15:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.071+0000 7fe95bfff700 1 -- 172.21.15.32:0/3028375270 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960019670 con 0x555db30f5630 2023-08-11T15:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.071+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/3028375270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 msgr2=0x555db30f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.071+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db30f5ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe960011680 tx=0x7fe960011990 comp rx=0 tx=0).stop 2023-08-11T15:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/3028375270 shutdown_connections 2023-08-11T15:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30fb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db3101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/3028375270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db30f5ab0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/3028375270 >> 172.21.15.32:0/3028375270 conn(0x555db30228f0 msgr2=0x555db30efe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/3028375270 shutdown_connections 2023-08-11T15:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/3028375270 wait complete. 2023-08-11T15:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 -- start start 2023-08-11T15:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.072+0000 7fe98b6a0b80 1 Processor -- start 2023-08-11T15:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db31049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30f5ff0 0x555db3104f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30fae80 0x555db30fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db2fe5a60 con 0x555db30f5ff0 2023-08-11T15:54:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db2fe5760 con 0x555db30fae80 2023-08-11T15:54:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db2ec98c0 con 0x555db30f5630 2023-08-11T15:54:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe98b6a0b80 10 client.0 _ll_register_callbacks cb 0x555db2fe8a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe975b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db31049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe975b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db31049f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:41110/0 (socket says 172.21.15.32:41110) 2023-08-11T15:54:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe975b9b700 1 -- 172.21.15.32:0/749482062 learned_addr learned my addr 172.21.15.32:0/749482062 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.073+0000 7fe97539a700 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30f5ff0 0x555db3104f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe97639c700 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30fae80 0x555db30fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe975b9b700 1 -- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30fae80 msgr2=0x555db30fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe975b9b700 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30fae80 0x555db30fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe975b9b700 1 -- 172.21.15.32:0/749482062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30f5ff0 msgr2=0x555db3104f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe975b9b700 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30f5ff0 0x555db3104f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe975b9b700 1 -- 172.21.15.32:0/749482062 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe960009e10 con 0x555db30f5630 2023-08-11T15:54:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe975b9b700 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db31049f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005340 tx=0x7fe96c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f81846a3a32e2807 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.074+0000 7fe9597fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:54:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe9597fa700 1 -- 172.21.15.32:0/749482062 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c004990 con 0x555db30f5630 2023-08-11T15:54:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe9597fa700 1 -- 172.21.15.32:0/749482062 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe96c02c040 con 0x555db30f5630 2023-08-11T15:54:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555db30e87e0 con 0x555db30f5630 2023-08-11T15:54:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe9597fa700 1 -- 172.21.15.32:0/749482062 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c004af0 con 0x555db30f5630 2023-08-11T15:54:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe98b6a0b80 10 client.26062 fetch_fsmap learned FSMap version 33 2023-08-11T15:54:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555db2e73d80 con 0x555db30f5630 2023-08-11T15:54:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe9597fa700 1 -- 172.21.15.32:0/749482062 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555db30e87e0 con 0x555db30f5630 2023-08-11T15:54:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.075+0000 7fe9597fa700 1 -- 172.21.15.32:0/749482062 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe96c030440 con 0x555db30f5630 2023-08-11T15:54:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe98b6a0b80 10 client.26062 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T15:54:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe98b6a0b80 10 client.26062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe98b6a0b80 20 client.26062 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db310ee20 0x555db3111240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe98b6a0b80 4 client.26062 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:54:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555db30c4480 con 0x555db310ee20 2023-08-11T15:54:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe97539a700 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db310ee20 0x555db3111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe97539a700 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db310ee20 0x555db3111240 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33a291f91ddc6a3 server_cookie=3b98b167dcec407e in_seq=0 out_seq=0 2023-08-11T15:54:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.076+0000 7fe9597fa700 10 client.26062 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.078+0000 7fe9597fa700 1 -- 172.21.15.32:0/749482062 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x555db30c4480 con 0x555db310ee20 2023-08-11T15:54:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.078+0000 7fe9597fa700 10 client.26062 handle_command_reply: tid=0 2023-08-11T15:54:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.078+0000 7fe98b6a0b80 1 client.26062 shutdown 2023-08-11T15:54:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.078+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 msgr2=0x555db31049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.078+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db31049f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005340 tx=0x7fe96c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 shutdown_connections 2023-08-11T15:54:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db310ee20 0x555db3111240 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30fae80 0x555db30fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db31049f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/749482062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30f5ff0 0x555db3104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 >> 172.21.15.32:0/749482062 conn(0x555db30228f0 msgr2=0x555db30ef5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe9597fa700 0 client.26062 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 shutdown_connections 2023-08-11T15:54:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/749482062 wait complete. 2023-08-11T15:54:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.079+0000 7fe98b6a0b80 20 client.26062 trim_cache size 0 max 16384 2023-08-11T15:54:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.095+0000 7fe98b6a0b80 1 Processor -- start 2023-08-11T15:54:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.095+0000 7fe98b6a0b80 1 -- start start 2023-08-11T15:54:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db30efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5ff0 0x555db30f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db30f9aa0 con 0x555db30fae80 2023-08-11T15:54:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db30f17d0 con 0x555db30f5ff0 2023-08-11T15:54:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db3111540 con 0x555db30f5630 2023-08-11T15:54:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe975b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db30efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe97539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5ff0 0x555db30f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe97539a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5ff0 0x555db30f8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:58452/0 (socket says 172.21.15.32:58452) 2023-08-11T15:54:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe97539a700 1 -- 172.21.15.32:0/769729777 learned_addr learned my addr 172.21.15.32:0/769729777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe975b9b700 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db30efd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:41120/0 (socket says 172.21.15.32:41120) 2023-08-11T15:54:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe97639c700 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe975b9b700 1 -- 172.21.15.32:0/769729777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5ff0 msgr2=0x555db30f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.096+0000 7fe975b9b700 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5ff0 0x555db30f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe975b9b700 1 -- 172.21.15.32:0/769729777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 msgr2=0x555db30f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe975b9b700 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe975b9b700 1 -- 172.21.15.32:0/769729777 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db30fb530 con 0x555db30f5630 2023-08-11T15:54:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe97639c700 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:54:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe975b9b700 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db30efd50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b6f0 tx=0x7fe96c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e76f240ac5958a10 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe964bff700 1 -- 172.21.15.32:0/769729777 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c030440 con 0x555db30f5630 2023-08-11T15:54:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe964bff700 1 -- 172.21.15.32:0/769729777 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96c01e040 con 0x555db30f5630 2023-08-11T15:54:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.097+0000 7fe964bff700 1 -- 172.21.15.32:0/769729777 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00f6c0 con 0x555db30f5630 2023-08-11T15:54:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/769729777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 msgr2=0x555db30efd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db30efd50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b6f0 tx=0x7fe96c005e80 comp rx=0 tx=0).stop 2023-08-11T15:54:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/769729777 shutdown_connections 2023-08-11T15:54:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db30f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5630 0x555db30efd50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/769729777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5ff0 0x555db30f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/769729777 >> 172.21.15.32:0/769729777 conn(0x555db30228f0 msgr2=0x555db3110e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/769729777 shutdown_connections 2023-08-11T15:54:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.098+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/769729777 wait complete. 2023-08-11T15:54:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 -- start start 2023-08-11T15:54:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 Processor -- start 2023-08-11T15:54:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db31024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db31029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db3102f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db3161ab0 con 0x555db30fae80 2023-08-11T15:54:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db3161bf0 con 0x555db30f5630 2023-08-11T15:54:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db3161d30 con 0x555db30f5ff0 2023-08-11T15:54:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.099+0000 7fe98b6a0b80 10 client.0 _ll_register_callbacks cb 0x7fe96c00bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:54:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db31029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97639c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db3102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe975b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db31024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db31029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:41122/0 (socket says 172.21.15.32:41122) 2023-08-11T15:54:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97639c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db3102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50628/0 (socket says 172.21.15.32:50628) 2023-08-11T15:54:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97639c700 1 -- 172.21.15.32:0/853819652 learned_addr learned my addr 172.21.15.32:0/853819652 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:54:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe975b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db31024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:58456/0 (socket says 172.21.15.32:58456) 2023-08-11T15:54:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 -- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 msgr2=0x555db31024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db31024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 -- 172.21.15.32:0/853819652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 msgr2=0x555db3102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:54:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db3102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 -- 172.21.15.32:0/853819652 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db2e73d80 con 0x555db30f5ff0 2023-08-11T15:54:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe975b9b700 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db31024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:54:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe97539a700 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db31029e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe9600092a0 tx=0x7fe960003c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49385f0402ebd4dc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:54:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.100+0000 7fe9597fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-08-11T15:54:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe9597fa700 1 -- 172.21.15.32:0/853819652 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600052c0 con 0x555db30f5ff0 2023-08-11T15:54:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe9597fa700 1 -- 172.21.15.32:0/853819652 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe960005420 con 0x555db30f5ff0 2023-08-11T15:54:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe960003680 con 0x555db30f5ff0 2023-08-11T15:54:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe9597fa700 1 -- 172.21.15.32:0/853819652 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9600055e0 con 0x555db30f5ff0 2023-08-11T15:54:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 10 client.26068 fetch_fsmap learned FSMap version 33 2023-08-11T15:54:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555db2e5cc00 con 0x555db30f5ff0 2023-08-11T15:54:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe9597fa700 1 -- 172.21.15.32:0/853819652 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe960003680 con 0x555db30f5ff0 2023-08-11T15:54:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe9597fa700 1 -- 172.21.15.32:0/853819652 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9600116b0 con 0x555db30f5ff0 2023-08-11T15:54:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T15:54:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 10 client.26068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:54:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 20 client.26068 populate_metadata read hostname 'smithi032' 2023-08-11T15:54:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db31633f0 0x555db3165810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:54:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 4 client.26068 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:54:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555db30407c0 con 0x555db31633f0 2023-08-11T15:54:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.101+0000 7fe975b9b700 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db31633f0 0x555db3165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:54:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.102+0000 7fe975b9b700 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db31633f0 0x555db3165810 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9681269b9b1af6 server_cookie=7ab7f7d41d01270d in_seq=0 out_seq=0 2023-08-11T15:54:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.102+0000 7fe9597fa700 10 client.26068 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.475+0000 7fe9597fa700 1 -- 172.21.15.32:0/853819652 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555db30407c0 con 0x555db31633f0 2023-08-11T15:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.475+0000 7fe9597fa700 10 client.26068 handle_command_reply: tid=0 2023-08-11T15:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 client.26068 shutdown 2023-08-11T15:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 msgr2=0x555db31029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db31029e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe9600092a0 tx=0x7fe960003c50 comp rx=0 tx=0).stop 2023-08-11T15:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 shutdown_connections 2023-08-11T15:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x555db31633f0 0x555db3165810 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555db30fae80 0x555db3102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe9597fa700 0 client.26068 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db30f5630 0x555db31024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 --2- 172.21.15.32:0/853819652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555db30f5ff0 0x555db31029e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 >> 172.21.15.32:0/853819652 conn(0x555db30228f0 msgr2=0x555db30f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 shutdown_connections 2023-08-11T15:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 1 -- 172.21.15.32:0/853819652 wait complete. 2023-08-11T15:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.476+0000 7fe98b6a0b80 20 client.26068 trim_cache size 0 max 16384 2023-08-11T15:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe95c0a00c0 msgr2=0x7fe95c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe95c0a00c0 0x7fe95c0a2580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b2e0 tx=0x7fe96c00b850 comp rx=0 tx=0).stop 2023-08-11T15:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 msgr2=0x7fe9701a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 0x7fe9701a4b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe96800efc0 tx=0x7fe96800c3a0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 shutdown_connections 2023-08-11T15:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fe95c0a00c0 0x7fe95c0a2580 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9701008a0 0x7fe970072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9701068b0 0x7fe9701a4b90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.477+0000 7fe977dff700 1 --2- 172.21.15.32:0/1206409612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe970101260 0x7fe970073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.478+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 >> 172.21.15.32:0/1206409612 conn(0x7fe9700fc370 msgr2=0x7fe9700fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.478+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 shutdown_connections 2023-08-11T15:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:54:49.478+0000 7fe977dff700 1 -- 172.21.15.32:0/1206409612 wait complete. 2023-08-11T15:54:49.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 914446 10.97 MB/sec execute 305 sec latency 292.016 ms 2023-08-11T15:54:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:50 smithi134 ceph-mon[130447]: pgmap v257: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2023-08-11T15:54:50.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 916511 10.98 MB/sec execute 306 sec latency 67.110 ms 2023-08-11T15:54:50.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:50 smithi032 ceph-mon[97103]: pgmap v257: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2023-08-11T15:54:50.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:50 smithi192 ceph-mon[130806]: pgmap v257: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2023-08-11T15:54:51.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 917477 11.02 MB/sec execute 307 sec latency 104.366 ms 2023-08-11T15:54:52.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:52 smithi134 ceph-mon[130447]: pgmap v258: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2023-08-11T15:54:52.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 919584 11.02 MB/sec execute 308 sec latency 32.597 ms 2023-08-11T15:54:52.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:52 smithi032 ceph-mon[97103]: pgmap v258: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2023-08-11T15:54:52.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:52 smithi192 ceph-mon[130806]: pgmap v258: 129 pgs: 129 active+clean; 79 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2023-08-11T15:54:53.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 921161 11.01 MB/sec execute 309 sec latency 83.101 ms 2023-08-11T15:54:54.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:54 smithi134 ceph-mon[130447]: pgmap v259: 129 pgs: 129 active+clean; 68 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2023-08-11T15:54:54.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 923614 11.02 MB/sec execute 310 sec latency 28.559 ms 2023-08-11T15:54:54.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:54 smithi032 ceph-mon[97103]: pgmap v259: 129 pgs: 129 active+clean; 68 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2023-08-11T15:54:54.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:54 smithi192 ceph-mon[130806]: pgmap v259: 129 pgs: 129 active+clean; 68 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2023-08-11T15:54:55.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 925384 11.01 MB/sec execute 311 sec latency 12.238 ms 2023-08-11T15:54:56.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:56 smithi134 ceph-mon[130447]: pgmap v260: 129 pgs: 129 active+clean; 68 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2023-08-11T15:54:56.794 INFO:tasks.workunit.client.0.smithi032.stdout: 1 927656 11.02 MB/sec execute 312 sec latency 53.866 ms 2023-08-11T15:54:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:56 smithi032 ceph-mon[97103]: pgmap v260: 129 pgs: 129 active+clean; 68 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2023-08-11T15:54:56.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:56 smithi192 ceph-mon[130806]: pgmap v260: 129 pgs: 129 active+clean; 68 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2023-08-11T15:54:57.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 929029 11.00 MB/sec execute 313 sec latency 49.306 ms 2023-08-11T15:54:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:58 smithi032 ceph-mon[97103]: pgmap v261: 129 pgs: 129 active+clean; 75 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2023-08-11T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:58 smithi192 ceph-mon[130806]: pgmap v261: 129 pgs: 129 active+clean; 75 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2023-08-11T15:54:58.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:58 smithi134 ceph-mon[130447]: pgmap v261: 129 pgs: 129 active+clean; 75 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2023-08-11T15:54:58.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 931279 11.01 MB/sec execute 314 sec latency 61.991 ms 2023-08-11T15:54:59.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:54:59 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:54:59.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:54:59 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:54:59.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:54:59 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:54:59.793 INFO:tasks.workunit.client.0.smithi032.stdout: 1 933132 11.00 MB/sec execute 315 sec latency 38.066 ms 2023-08-11T15:55:00.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:00 smithi032 ceph-mon[97103]: pgmap v262: 129 pgs: 129 active+clean; 76 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2023-08-11T15:55:00.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:00 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:55:00.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:00 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:55:00.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:00 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:55:00.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:00 smithi192 ceph-mon[130806]: pgmap v262: 129 pgs: 129 active+clean; 76 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2023-08-11T15:55:00.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:00 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:55:00.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:00 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:55:00.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:00 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:55:00.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:00 smithi134 ceph-mon[130447]: pgmap v262: 129 pgs: 129 active+clean; 76 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2023-08-11T15:55:00.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:00 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:55:00.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:00 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:55:00.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:00 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:55:00.794 INFO:tasks.workunit.client.0.smithi032.stdout: 1 935232 11.00 MB/sec execute 316 sec latency 15.015 ms 2023-08-11T15:55:01.794 INFO:tasks.workunit.client.0.smithi032.stdout: 1 935464 10.97 MB/sec execute 317 sec latency 201.138 ms 2023-08-11T15:55:02.562 INFO:tasks.workunit.client.0.smithi032.stdout:[936736] write failed on handle 9926 (Resource temporarily unavailable) 2023-08-11T15:55:02.562 INFO:tasks.workunit.client.0.smithi032.stdout:Child failed with status 1 2023-08-11T15:55:02.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T15:55:02.564 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-08-11T15:55:02.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-08-11T15:55:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 -- 172.21.15.32:0/1763211218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c108ba0 msgr2=0x7f9c3c108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 --2- 172.21.15.32:0/1763211218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c108ba0 0x7f9c3c108f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0098f0 tx=0x7f9c2c009c00 comp rx=0 tx=0).stop 2023-08-11T15:55:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 -- 172.21.15.32:0/1763211218 shutdown_connections 2023-08-11T15:55:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 --2- 172.21.15.32:0/1763211218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c1034c0 0x7f9c3c10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 --2- 172.21.15.32:0/1763211218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c108ba0 0x7f9c3c108f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 --2- 172.21.15.32:0/1763211218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c3c102b00 0x7f9c3c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 -- 172.21.15.32:0/1763211218 >> 172.21.15.32:0/1763211218 conn(0x7f9c3c0fe5d0 msgr2=0x7f9c3c1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:55:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 -- 172.21.15.32:0/1763211218 shutdown_connections 2023-08-11T15:55:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.636+0000 7f9c411ff700 1 -- 172.21.15.32:0/1763211218 wait complete. 2023-08-11T15:55:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 Processor -- start 2023-08-11T15:55:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 -- start start 2023-08-11T15:55:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c102b00 0x7f9c3c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c1034c0 0x7f9c3c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c3c108ba0 0x7f9c3c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c1990b0 con 0x7f9c3c108ba0 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c199220 con 0x7f9c3c102b00 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.637+0000 7f9c411ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c1993e0 con 0x7f9c3c1034c0 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3a59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c1034c0 0x7f9c3c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c102b00 0x7f9c3c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3a59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c1034c0 0x7f9c3c19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38802/0 (socket says 172.21.15.32:38802) 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c102b00 0x7f9c3c19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55966/0 (socket says 172.21.15.32:55966) 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3a59c700 1 -- 172.21.15.32:0/2125373903 learned_addr learned my addr 172.21.15.32:0/2125373903 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:55:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c3c108ba0 0x7f9c3c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 -- 172.21.15.32:0/2125373903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c1034c0 msgr2=0x7f9c3c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c1034c0 0x7f9c3c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 -- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c3c108ba0 msgr2=0x7f9c3c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c3c108ba0 0x7f9c3c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 -- 172.21.15.32:0/2125373903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c009520 con 0x7f9c3c102b00 2023-08-11T15:55:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c3c108ba0 0x7f9c3c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:55:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.638+0000 7f9c3ad9d700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c102b00 0x7f9c3c19ebe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009810 tx=0x7f9c2c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:55:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.639+0000 7f9c23fff700 1 -- 172.21.15.32:0/2125373903 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c01c070 con 0x7f9c3c102b00 2023-08-11T15:55:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.639+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c199660 con 0x7f9c3c102b00 2023-08-11T15:55:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.639+0000 7f9c23fff700 1 -- 172.21.15.32:0/2125373903 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c2c005040 con 0x7f9c3c102b00 2023-08-11T15:55:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.639+0000 7f9c23fff700 1 -- 172.21.15.32:0/2125373903 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c003970 con 0x7f9c3c102b00 2023-08-11T15:55:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.639+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c1a5bd0 con 0x7f9c3c102b00 2023-08-11T15:55:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.643+0000 7f9c23fff700 1 -- 172.21.15.32:0/2125373903 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f9c2c020030 con 0x7f9c3c102b00 2023-08-11T15:55:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.644+0000 7f9c54ac8b80 1 Processor -- start 2023-08-11T15:55:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c54ac8b80 1 -- start start 2023-08-11T15:55:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136630 0x558ecc136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136ff0 0x558ecc142c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc08f5d0 con 0x558ecc136630 2023-08-11T15:55:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc08cef0 con 0x558ecc136ff0 2023-08-11T15:55:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc0887e0 con 0x558ecc13be80 2023-08-11T15:55:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38808/0 (socket says 172.21.15.32:38808) 2023-08-11T15:55:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136630 0x558ecc136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3b59e700 1 -- 172.21.15.32:0/2770287882 learned_addr learned my addr 172.21.15.32:0/2770287882 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:55:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3a59c700 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136ff0 0x558ecc142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3b59e700 1 -- 172.21.15.32:0/2770287882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136ff0 msgr2=0x558ecc142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136ff0 0x558ecc142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.645+0000 7f9c3b59e700 1 -- 172.21.15.32:0/2770287882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136630 msgr2=0x558ecc136ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136630 0x558ecc136ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c3b59e700 1 -- 172.21.15.32:0/2770287882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ecbf085d0 con 0x558ecc13be80 2023-08-11T15:55:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c3ad9d700 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136630 0x558ecc136ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:55:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c23fff700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9c240a00c0 0x7f9c240a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13c260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c340093e0 tx=0x7f9c34009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02dea0b2c0cae35 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:55:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c23fff700 1 -- 172.21.15.32:0/2125373903 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9c2c0cc820 con 0x7f9c3c102b00 2023-08-11T15:55:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c217fa700 1 -- 172.21.15.32:0/2770287882 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400e070 con 0x558ecc13be80 2023-08-11T15:55:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c217fa700 1 -- 172.21.15.32:0/2770287882 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c3400f040 con 0x558ecc13be80 2023-08-11T15:55:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c3ad9d700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9c240a00c0 0x7f9c240a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c217fa700 1 -- 172.21.15.32:0/2770287882 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34015440 con 0x558ecc13be80 2023-08-11T15:55:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/2770287882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 msgr2=0x558ecc13c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c3ad9d700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9c240a00c0 0x7f9c240a2580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0c9d10 tx=0x7f9c2c009c00 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:55:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.647+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13c260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c340093e0 tx=0x7f9c34009920 comp rx=0 tx=0).stop 2023-08-11T15:55:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/2770287882 shutdown_connections 2023-08-11T15:55:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136ff0 0x558ecc142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13c260 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/2770287882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136630 0x558ecc136ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/2770287882 >> 172.21.15.32:0/2770287882 conn(0x558ecc0638f0 msgr2=0x558ecc130e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:55:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/2770287882 shutdown_connections 2023-08-11T15:55:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/2770287882 wait complete. 2023-08-11T15:55:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.648+0000 7f9c54ac8b80 1 -- start start 2023-08-11T15:55:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 1 Processor -- start 2023-08-11T15:55:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc146050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc026a60 con 0x558ecc136ff0 2023-08-11T15:55:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc026760 con 0x558ecc136630 2023-08-11T15:55:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecbf0a8c0 con 0x558ecc13be80 2023-08-11T15:55:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c54ac8b80 10 client.0 _ll_register_callbacks cb 0x558ecc029a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:55:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:55988/0 (socket says 172.21.15.32:55988) 2023-08-11T15:55:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39858/0 (socket says 172.21.15.32:39858) 2023-08-11T15:55:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.649+0000 7f9c3b59e700 1 -- 172.21.15.32:0/157049087 learned_addr learned my addr 172.21.15.32:0/157049087 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:55:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c3a59c700 1 -- 172.21.15.32:0/157049087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 msgr2=0x558ecc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c3a59c700 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c3a59c700 1 -- 172.21.15.32:0/157049087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 msgr2=0x558ecc145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c3a59c700 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c3a59c700 1 -- 172.21.15.32:0/157049087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c34009090 con 0x558ecc136ff0 2023-08-11T15:55:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c3a59c700 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc146050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c100ff0 tx=0x7f9c30012af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca3780c4db06e53 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:55:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:55:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.650+0000 7f9c1eff5700 1 -- 172.21.15.32:0/157049087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30017070 con 0x558ecc136ff0 2023-08-11T15:55:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ecc1297e0 con 0x558ecc136ff0 2023-08-11T15:55:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c1eff5700 1 -- 172.21.15.32:0/157049087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9c30004020 con 0x558ecc136ff0 2023-08-11T15:55:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c1eff5700 1 -- 172.21.15.32:0/157049087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c300122e0 con 0x558ecc136ff0 2023-08-11T15:55:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c54ac8b80 10 client.16044 fetch_fsmap learned FSMap version 33 2023-08-11T15:55:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558ecbeb4d80 con 0x558ecc136ff0 2023-08-11T15:55:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c1eff5700 1 -- 172.21.15.32:0/157049087 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ecc1297e0 con 0x558ecc136ff0 2023-08-11T15:55:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c1eff5700 1 -- 172.21.15.32:0/157049087 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c3000f4e0 con 0x558ecc136ff0 2023-08-11T15:55:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c54ac8b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T15:55:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c54ac8b80 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:55:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.651+0000 7f9c54ac8b80 20 client.16044 populate_metadata read hostname 'smithi032' 2023-08-11T15:55:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.652+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc14ff40 0x558ecc152360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.652+0000 7f9c54ac8b80 4 client.16044 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T15:55:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.652+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ecc105480 con 0x558ecc14ff40 2023-08-11T15:55:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.652+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc14ff40 0x558ecc152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.652+0000 7f9c1eff5700 10 client.16044 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:55:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.652+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc14ff40 0x558ecc152360 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2725adad18c46676 server_cookie=87e09bd836aeeb5b in_seq=0 out_seq=0 2023-08-11T15:55:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.653+0000 7f9c1eff5700 1 -- 172.21.15.32:0/157049087 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558ecc105480 con 0x558ecc14ff40 2023-08-11T15:55:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.653+0000 7f9c1eff5700 10 client.16044 handle_command_reply: tid=0 2023-08-11T15:55:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.653+0000 7f9c54ac8b80 1 client.16044 shutdown 2023-08-11T15:55:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.653+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 msgr2=0x558ecc146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.653+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc146050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c100ff0 tx=0x7f9c30012af0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 shutdown_connections 2023-08-11T15:55:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc14ff40 0x558ecc152360 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/157049087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc146050 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 >> 172.21.15.32:0/157049087 conn(0x558ecc0638f0 msgr2=0x558ecc1305d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:55:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c1eff5700 0 client.16044 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:55:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 shutdown_connections 2023-08-11T15:55:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/157049087 wait complete. 2023-08-11T15:55:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.654+0000 7f9c54ac8b80 20 client.16044 trim_cache size 0 max 16384 2023-08-11T15:55:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.674+0000 7f9c54ac8b80 1 Processor -- start 2023-08-11T15:55:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.674+0000 7f9c54ac8b80 1 -- start start 2023-08-11T15:55:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc13da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc145630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc130d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc14d640 con 0x558ecc136ff0 2023-08-11T15:55:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc13e3f0 con 0x558ecc136630 2023-08-11T15:55:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc13abe0 con 0x558ecc13be80 2023-08-11T15:55:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc13da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc13da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:56002/0 (socket says 172.21.15.32:56002) 2023-08-11T15:55:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 -- 172.21.15.32:0/1013311979 learned_addr learned my addr 172.21.15.32:0/1013311979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:55:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc130d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 -- 172.21.15.32:0/1013311979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 msgr2=0x558ecc130d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc130d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 -- 172.21.15.32:0/1013311979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 msgr2=0x558ecc145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.675+0000 7f9c3b59e700 1 -- 172.21.15.32:0/1013311979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ecbe9dc00 con 0x558ecc136630 2023-08-11T15:55:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.676+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc13da00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007b70 tx=0x7f9c34009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=938ee203c4f7c40f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:55:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.676+0000 7f9c21ffb700 1 -- 172.21.15.32:0/1013311979 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400e040 con 0x558ecc136630 2023-08-11T15:55:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.676+0000 7f9c21ffb700 1 -- 172.21.15.32:0/1013311979 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c34027070 con 0x558ecc136630 2023-08-11T15:55:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.676+0000 7f9c21ffb700 1 -- 172.21.15.32:0/1013311979 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34003c70 con 0x558ecc136630 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/1013311979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 msgr2=0x558ecc13da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc13da00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007b70 tx=0x7f9c34009ef0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/1013311979 shutdown_connections 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc13be80 0x558ecc130d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc136630 0x558ecc13da00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/1013311979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/1013311979 >> 172.21.15.32:0/1013311979 conn(0x558ecc0638f0 msgr2=0x558ecc140800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:55:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/1013311979 shutdown_connections 2023-08-11T15:55:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/1013311979 wait complete. 2023-08-11T15:55:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.677+0000 7f9c54ac8b80 1 -- start start 2023-08-11T15:55:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.678+0000 7f9c54ac8b80 1 Processor -- start 2023-08-11T15:55:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.678+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc136630 0x558ecc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.678+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.678+0000 7f9c54ac8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc13be80 0x558ecc198600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.678+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc1a2cf0 con 0x558ecc136ff0 2023-08-11T15:55:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.678+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc1a2e30 con 0x558ecc13be80 2023-08-11T15:55:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.678+0000 7f9c54ac8b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ecc1a2ff0 con 0x558ecc136630 2023-08-11T15:55:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c54ac8b80 10 client.0 _ll_register_callbacks cb 0x7f9c30009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T15:55:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc136630 0x558ecc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39882/0 (socket says 172.21.15.32:39882) 2023-08-11T15:55:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 -- 172.21.15.32:0/4005424437 learned_addr learned my addr 172.21.15.32:0/4005424437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3b59e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc136630 0x558ecc197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38822/0 (socket says 172.21.15.32:38822) 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc13be80 0x558ecc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 -- 172.21.15.32:0/4005424437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc136630 msgr2=0x558ecc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc136630 0x558ecc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 -- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc13be80 msgr2=0x558ecc198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc13be80 0x558ecc198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 -- 172.21.15.32:0/4005424437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ecbf085d0 con 0x558ecc136ff0 2023-08-11T15:55:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc136630 0x558ecc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:55:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c3a59c700 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc1980c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x558ecc132690 tx=0x7f9c3000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b00070691de4c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:55:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.679+0000 7f9c1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-08-11T15:55:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c1eff5700 1 -- 172.21.15.32:0/4005424437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30017070 con 0x558ecc136ff0 2023-08-11T15:55:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c3400f040 con 0x558ecc136ff0 2023-08-11T15:55:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c1eff5700 1 -- 172.21.15.32:0/4005424437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f9c30005280 con 0x558ecc136ff0 2023-08-11T15:55:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c1eff5700 1 -- 172.21.15.32:0/4005424437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c30005440 con 0x558ecc136ff0 2023-08-11T15:55:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 10 client.16050 fetch_fsmap learned FSMap version 33 2023-08-11T15:55:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558ecbedb1f0 con 0x558ecc136ff0 2023-08-11T15:55:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c1eff5700 1 -- 172.21.15.32:0/4005424437 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c3400f040 con 0x558ecc136ff0 2023-08-11T15:55:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c1eff5700 1 -- 172.21.15.32:0/4005424437 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c3001c680 con 0x558ecc136ff0 2023-08-11T15:55:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T15:55:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T15:55:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 20 client.16050 populate_metadata read hostname 'smithi032' 2023-08-11T15:55:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc1a47a0 0x558ecc1a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:55:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 4 client.16050 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-08-11T15:55:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.680+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 --> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ecc0817c0 con 0x558ecc1a47a0 2023-08-11T15:55:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.681+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc1a47a0 0x558ecc1a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:55:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.681+0000 7f9c3b59e700 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc1a47a0 0x558ecc1a6bc0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d6096a40b0a1c6 server_cookie=b6af868aa0988c1d in_seq=0 out_seq=0 2023-08-11T15:55:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:02.681+0000 7f9c1eff5700 10 client.16050 ms_handle_connect on v2:172.21.15.134:6838/945169036 2023-08-11T15:55:02.699 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:02 smithi032 ceph-mon[97103]: pgmap v263: 129 pgs: 129 active+clean; 76 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 337 op/s 2023-08-11T15:55:02.725 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:02 smithi134 ceph-mon[130447]: pgmap v263: 129 pgs: 129 active+clean; 76 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 337 op/s 2023-08-11T15:55:02.844 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:02 smithi192 ceph-mon[130806]: pgmap v263: 129 pgs: 129 active+clean; 76 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 337 op/s 2023-08-11T15:55:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.204+0000 7f9c1eff5700 1 -- 172.21.15.32:0/4005424437 <== mds.0 v2:172.21.15.134:6838/945169036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ecc0817c0 con 0x558ecc1a47a0 2023-08-11T15:55:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.204+0000 7f9c1eff5700 10 client.16050 handle_command_reply: tid=0 2023-08-11T15:55:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.204+0000 7f9c54ac8b80 1 client.16050 shutdown 2023-08-11T15:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 msgr2=0x558ecc1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc1980c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x558ecc132690 tx=0x7f9c3000fd40 comp rx=0 tx=0).stop 2023-08-11T15:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 shutdown_connections 2023-08-11T15:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:6838/945169036,v1:172.21.15.134:6839/945169036] conn(0x558ecc1a47a0 0x558ecc1a6bc0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558ecc13be80 0x558ecc198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ecc136630 0x558ecc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 --2- 172.21.15.32:0/4005424437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ecc136ff0 0x558ecc1980c0 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x558ecc132690 tx=0x7f9c3000fd40 comp rx=0 tx=0).stop 2023-08-11T15:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 >> 172.21.15.32:0/4005424437 conn(0x558ecc0638f0 msgr2=0x558ecc13a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:55:03.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.209+0000 7f9c1eff5700 0 client.16050 ms_handle_reset on v2:172.21.15.134:6838/945169036 2023-08-11T15:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.213+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 shutdown_connections 2023-08-11T15:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.214+0000 7f9c54ac8b80 1 -- 172.21.15.32:0/4005424437 wait complete. 2023-08-11T15:55:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.218+0000 7f9c54ac8b80 20 client.16050 trim_cache size 0 max 16384 2023-08-11T15:55:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-08-11T15:55:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-08-11T15:55:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-08-11T15:55:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-08-11T15:55:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.230+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9c240a00c0 msgr2=0x7f9c240a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.230+0000 7f9c411ff700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9c240a00c0 0x7f9c240a2580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0c9d10 tx=0x7f9c2c009c00 comp rx=0 tx=0).stop 2023-08-11T15:55:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.230+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c102b00 msgr2=0x7f9c3c19ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:55:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.230+0000 7f9c411ff700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c102b00 0x7f9c3c19ebe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009810 tx=0x7f9c2c00fa00 comp rx=0 tx=0).stop 2023-08-11T15:55:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.234+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 shutdown_connections 2023-08-11T15:55:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.234+0000 7f9c411ff700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f9c240a00c0 0x7f9c240a2580 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.234+0000 7f9c411ff700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c3c108ba0 0x7f9c3c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.234+0000 7f9c411ff700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c3c102b00 0x7f9c3c19ebe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.234+0000 7f9c411ff700 1 --2- 172.21.15.32:0/2125373903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c1034c0 0x7f9c3c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:55:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.234+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 >> 172.21.15.32:0/2125373903 conn(0x7f9c3c0fe5d0 msgr2=0x7f9c3c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:55:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.245+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 shutdown_connections 2023-08-11T15:55:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-08-11T15:55:03.250+0000 7f9c411ff700 1 -- 172.21.15.32:0/2125373903 wait complete. 2023-08-11T15:55:04.234 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92684807f016ce045ac66e37fdde6f8c47857bcc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-08-11T15:55:04.374 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=40ccea2273d247b0812d9d3cc46113ea Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92684807f016ce045ac66e37fdde6f8c47857bcc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-08-11T15:55:04.377 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-08-11T15:55:04.389 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:55:04.394 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:04 smithi192 ceph-mon[130806]: osdmap e104: 12 total, 12 up, 12 in 2023-08-11T15:55:04.394 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:04 smithi192 ceph-mon[130806]: pgmap v265: 129 pgs: 129 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-08-11T15:55:04.394 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:04 smithi032 ceph-mon[97103]: osdmap e104: 12 total, 12 up, 12 in 2023-08-11T15:55:04.395 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:04 smithi032 ceph-mon[97103]: pgmap v265: 129 pgs: 129 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-08-11T15:55:04.413 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-08-11T15:55:04.424 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-08-11T15:55:04.425 INFO:teuthology.task.exec:Executing custom commands... 2023-08-11T15:55:04.425 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:55:04.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-08-11T15:55:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:04 smithi134 ceph-mon[130447]: osdmap e104: 12 total, 12 up, 12 in 2023-08-11T15:55:04.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:04 smithi134 ceph-mon[130447]: pgmap v265: 129 pgs: 129 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-08-11T15:55:04.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-08-11T15:55:06.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:06 smithi134 ceph-mon[130447]: pgmap v266: 129 pgs: 129 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-08-11T15:55:06.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:06 smithi032 ceph-mon[97103]: pgmap v266: 129 pgs: 129 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-08-11T15:55:06.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:06 smithi192 ceph-mon[130806]: pgmap v266: 129 pgs: 129 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-08-11T15:55:08.305 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:08 smithi032 ceph-mon[97103]: pgmap v267: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-08-11T15:55:08.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:08 smithi192 ceph-mon[130806]: pgmap v267: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-08-11T15:55:08.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:08 smithi134 ceph-mon[130447]: pgmap v267: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-08-11T15:55:10.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:10 smithi134 ceph-mon[130447]: pgmap v268: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2023-08-11T15:55:10.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:10 smithi032 ceph-mon[97103]: pgmap v268: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2023-08-11T15:55:10.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:10 smithi192 ceph-mon[130806]: pgmap v268: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2023-08-11T15:55:12.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:12 smithi134 ceph-mon[130447]: pgmap v269: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2023-08-11T15:55:12.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:12 smithi032 ceph-mon[97103]: pgmap v269: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2023-08-11T15:55:12.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:12 smithi192 ceph-mon[130806]: pgmap v269: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2023-08-11T15:55:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:14 smithi134 ceph-mon[130447]: pgmap v270: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.8 MiB/s wr, 227 op/s 2023-08-11T15:55:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:14 smithi032 ceph-mon[97103]: pgmap v270: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.8 MiB/s wr, 227 op/s 2023-08-11T15:55:14.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:14 smithi192 ceph-mon[130806]: pgmap v270: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.8 MiB/s wr, 227 op/s 2023-08-11T15:55:16.675 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:16 smithi134 ceph-mon[130447]: pgmap v271: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.6 MiB/s wr, 194 op/s 2023-08-11T15:55:16.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:16 smithi032 ceph-mon[97103]: pgmap v271: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.6 MiB/s wr, 194 op/s 2023-08-11T15:55:16.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:16 smithi192 ceph-mon[130806]: pgmap v271: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.6 MiB/s wr, 194 op/s 2023-08-11T15:55:18.039 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:18 smithi032 ceph-mon[97103]: osdmap e105: 12 total, 12 up, 12 in 2023-08-11T15:55:18.040 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:18 smithi032 ceph-mon[97103]: pgmap v273: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 133 op/s 2023-08-11T15:55:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:18 smithi192 ceph-mon[130806]: osdmap e105: 12 total, 12 up, 12 in 2023-08-11T15:55:18.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:18 smithi192 ceph-mon[130806]: pgmap v273: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 133 op/s 2023-08-11T15:55:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:18 smithi134 ceph-mon[130447]: osdmap e105: 12 total, 12 up, 12 in 2023-08-11T15:55:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:18 smithi134 ceph-mon[130447]: pgmap v273: 129 pgs: 129 active+clean; 72 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 133 op/s 2023-08-11T15:55:20.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:20 smithi134 ceph-mon[130447]: pgmap v274: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2023-08-11T15:55:20.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:20 smithi032 ceph-mon[97103]: pgmap v274: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2023-08-11T15:55:20.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:20 smithi192 ceph-mon[130806]: pgmap v274: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2023-08-11T15:55:22.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:22 smithi134 ceph-mon[130447]: pgmap v275: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2023-08-11T15:55:22.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:22 smithi032 ceph-mon[97103]: pgmap v275: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2023-08-11T15:55:22.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:22 smithi192 ceph-mon[130806]: pgmap v275: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2023-08-11T15:55:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:24 smithi134 ceph-mon[130447]: pgmap v276: 129 pgs: 129 active+clean; 82 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.2 MiB/s wr, 96 op/s 2023-08-11T15:55:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:24 smithi032 ceph-mon[97103]: pgmap v276: 129 pgs: 129 active+clean; 82 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.2 MiB/s wr, 96 op/s 2023-08-11T15:55:24.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:24 smithi192 ceph-mon[130806]: pgmap v276: 129 pgs: 129 active+clean; 82 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.2 MiB/s wr, 96 op/s 2023-08-11T15:55:26.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:26 smithi134 ceph-mon[130447]: pgmap v277: 129 pgs: 129 active+clean; 82 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.2 MiB/s wr, 96 op/s 2023-08-11T15:55:26.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:26 smithi032 ceph-mon[97103]: pgmap v277: 129 pgs: 129 active+clean; 82 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.2 MiB/s wr, 96 op/s 2023-08-11T15:55:26.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:26 smithi192 ceph-mon[130806]: pgmap v277: 129 pgs: 129 active+clean; 82 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.2 MiB/s wr, 96 op/s 2023-08-11T15:55:28.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:28 smithi032 ceph-mon[97103]: pgmap v278: 129 pgs: 129 active+clean; 84 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 3.8 MiB/s wr, 119 op/s 2023-08-11T15:55:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:28 smithi192 ceph-mon[130806]: pgmap v278: 129 pgs: 129 active+clean; 84 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 3.8 MiB/s wr, 119 op/s 2023-08-11T15:55:28.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:28 smithi134 ceph-mon[130447]: pgmap v278: 129 pgs: 129 active+clean; 84 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 3.8 MiB/s wr, 119 op/s 2023-08-11T15:55:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:30 smithi134 ceph-mon[130447]: pgmap v279: 129 pgs: 129 active+clean; 87 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.1 MiB/s wr, 88 op/s 2023-08-11T15:55:30.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:30 smithi032 ceph-mon[97103]: pgmap v279: 129 pgs: 129 active+clean; 87 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.1 MiB/s wr, 88 op/s 2023-08-11T15:55:30.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:30 smithi192 ceph-mon[130806]: pgmap v279: 129 pgs: 129 active+clean; 87 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.1 MiB/s wr, 88 op/s 2023-08-11T15:55:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:32 smithi134 ceph-mon[130447]: pgmap v280: 129 pgs: 129 active+clean; 87 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2023-08-11T15:55:32.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:32 smithi032 ceph-mon[97103]: pgmap v280: 129 pgs: 129 active+clean; 87 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2023-08-11T15:55:32.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:32 smithi192 ceph-mon[130806]: pgmap v280: 129 pgs: 129 active+clean; 87 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2023-08-11T15:55:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:34 smithi134 ceph-mon[130447]: pgmap v281: 129 pgs: 129 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.3 MiB/s wr, 108 op/s 2023-08-11T15:55:34.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:34 smithi032 ceph-mon[97103]: pgmap v281: 129 pgs: 129 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.3 MiB/s wr, 108 op/s 2023-08-11T15:55:34.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:34 smithi192 ceph-mon[130806]: pgmap v281: 129 pgs: 129 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.3 MiB/s wr, 108 op/s 2023-08-11T15:55:36.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:36 smithi134 ceph-mon[130447]: pgmap v282: 129 pgs: 129 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2023-08-11T15:55:36.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:36 smithi032 ceph-mon[97103]: pgmap v282: 129 pgs: 129 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2023-08-11T15:55:36.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:36 smithi192 ceph-mon[130806]: pgmap v282: 129 pgs: 129 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2023-08-11T15:55:38.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:38 smithi032 ceph-mon[97103]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2023-08-11T15:55:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:38 smithi192 ceph-mon[130806]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2023-08-11T15:55:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:38 smithi134 ceph-mon[130447]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2023-08-11T15:55:40.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:40 smithi134 ceph-mon[130447]: pgmap v284: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 911 KiB/s wr, 67 op/s 2023-08-11T15:55:40.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:40 smithi032 ceph-mon[97103]: pgmap v284: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 911 KiB/s wr, 67 op/s 2023-08-11T15:55:40.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:40 smithi192 ceph-mon[130806]: pgmap v284: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 911 KiB/s wr, 67 op/s 2023-08-11T15:55:42.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:42 smithi134 ceph-mon[130447]: pgmap v285: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 558 KiB/s wr, 57 op/s 2023-08-11T15:55:42.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:42 smithi032 ceph-mon[97103]: pgmap v285: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 558 KiB/s wr, 57 op/s 2023-08-11T15:55:42.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:42 smithi192 ceph-mon[130806]: pgmap v285: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 558 KiB/s wr, 57 op/s 2023-08-11T15:55:44.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:44 smithi134 ceph-mon[130447]: pgmap v286: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 623 KiB/s wr, 61 op/s 2023-08-11T15:55:44.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:44 smithi032 ceph-mon[97103]: pgmap v286: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 623 KiB/s wr, 61 op/s 2023-08-11T15:55:44.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:44 smithi192 ceph-mon[130806]: pgmap v286: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 623 KiB/s wr, 61 op/s 2023-08-11T15:55:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:46 smithi134 ceph-mon[130447]: pgmap v287: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 94 KiB/s wr, 26 op/s 2023-08-11T15:55:46.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:46 smithi032 ceph-mon[97103]: pgmap v287: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 94 KiB/s wr, 26 op/s 2023-08-11T15:55:46.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:46 smithi192 ceph-mon[130806]: pgmap v287: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 94 KiB/s wr, 26 op/s 2023-08-11T15:55:48.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:48 smithi032 ceph-mon[97103]: pgmap v288: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 94 KiB/s wr, 26 op/s 2023-08-11T15:55:48.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:48 smithi192 ceph-mon[130806]: pgmap v288: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 94 KiB/s wr, 26 op/s 2023-08-11T15:55:48.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:48 smithi134 ceph-mon[130447]: pgmap v288: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 94 KiB/s wr, 26 op/s 2023-08-11T15:55:50.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:50 smithi134 ceph-mon[130447]: pgmap v289: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 95 KiB/s wr, 13 op/s 2023-08-11T15:55:50.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:50 smithi032 ceph-mon[97103]: pgmap v289: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 95 KiB/s wr, 13 op/s 2023-08-11T15:55:50.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:50 smithi192 ceph-mon[130806]: pgmap v289: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 95 KiB/s wr, 13 op/s 2023-08-11T15:55:52.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:52 smithi134 ceph-mon[130447]: pgmap v290: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 74 KiB/s wr, 5 op/s 2023-08-11T15:55:52.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:52 smithi032 ceph-mon[97103]: pgmap v290: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 74 KiB/s wr, 5 op/s 2023-08-11T15:55:52.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:52 smithi192 ceph-mon[130806]: pgmap v290: 129 pgs: 129 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 74 KiB/s wr, 5 op/s 2023-08-11T15:55:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:54 smithi134 ceph-mon[130447]: pgmap v291: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 82 KiB/s wr, 7 op/s 2023-08-11T15:55:54.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:54 smithi032 ceph-mon[97103]: pgmap v291: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 82 KiB/s wr, 7 op/s 2023-08-11T15:55:54.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:54 smithi192 ceph-mon[130806]: pgmap v291: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 82 KiB/s wr, 7 op/s 2023-08-11T15:55:56.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:56 smithi134 ceph-mon[130447]: pgmap v292: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 2 op/s 2023-08-11T15:55:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:56 smithi032 ceph-mon[97103]: pgmap v292: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 2 op/s 2023-08-11T15:55:56.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:56 smithi192 ceph-mon[130806]: pgmap v292: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 2 op/s 2023-08-11T15:55:58.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:55:58 smithi032 ceph-mon[97103]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-08-11T15:55:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:55:58 smithi192 ceph-mon[130806]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-08-11T15:55:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:55:58 smithi134 ceph-mon[130447]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-08-11T15:56:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:00 smithi134 ceph-mon[130447]: pgmap v294: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-08-11T15:56:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:00 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:56:00.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:00 smithi032 ceph-mon[97103]: pgmap v294: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-08-11T15:56:00.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:00 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:56:00.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:00 smithi192 ceph-mon[130806]: pgmap v294: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-08-11T15:56:00.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:00 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:56:01.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:01 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:56:01.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:01 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:56:01.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:01 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:56:01.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:01 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:56:01.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:01 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:56:01.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:01 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:56:01.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:01 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:56:01.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:01 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:56:01.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:01 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:56:02.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:02 smithi134 ceph-mon[130447]: pgmap v295: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.6 KiB/s wr, 2 op/s 2023-08-11T15:56:02.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:02 smithi032 ceph-mon[97103]: pgmap v295: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.6 KiB/s wr, 2 op/s 2023-08-11T15:56:02.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:02 smithi192 ceph-mon[130806]: pgmap v295: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.6 KiB/s wr, 2 op/s 2023-08-11T15:56:04.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:04 smithi134 ceph-mon[130447]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 59 KiB/s wr, 18 op/s 2023-08-11T15:56:04.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:04 smithi032 ceph-mon[97103]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 59 KiB/s wr, 18 op/s 2023-08-11T15:56:04.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:04 smithi192 ceph-mon[130806]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 59 KiB/s wr, 18 op/s 2023-08-11T15:56:06.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:06 smithi134 ceph-mon[130447]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 KiB/s wr, 17 op/s 2023-08-11T15:56:06.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:06 smithi032 ceph-mon[97103]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 KiB/s wr, 17 op/s 2023-08-11T15:56:06.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:06 smithi192 ceph-mon[130806]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 KiB/s wr, 17 op/s 2023-08-11T15:56:08.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:08 smithi134 ceph-mon[130447]: pgmap v298: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 KiB/s wr, 17 op/s 2023-08-11T15:56:08.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:08 smithi134 ceph-mon[130447]: osdmap e106: 12 total, 12 up, 12 in 2023-08-11T15:56:08.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:08 smithi032 ceph-mon[97103]: pgmap v298: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 KiB/s wr, 17 op/s 2023-08-11T15:56:08.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:08 smithi032 ceph-mon[97103]: osdmap e106: 12 total, 12 up, 12 in 2023-08-11T15:56:08.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:08 smithi192 ceph-mon[130806]: pgmap v298: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 KiB/s wr, 17 op/s 2023-08-11T15:56:08.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:08 smithi192 ceph-mon[130806]: osdmap e106: 12 total, 12 up, 12 in 2023-08-11T15:56:10.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:10 smithi032 ceph-mon[97103]: pgmap v300: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 20 op/s 2023-08-11T15:56:10.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:10 smithi192 ceph-mon[130806]: pgmap v300: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 20 op/s 2023-08-11T15:56:10.970 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:10 smithi134 ceph-mon[130447]: pgmap v300: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 20 op/s 2023-08-11T15:56:12.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:12 smithi032 ceph-mon[97103]: pgmap v301: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 20 op/s 2023-08-11T15:56:12.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:12 smithi192 ceph-mon[130806]: pgmap v301: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 20 op/s 2023-08-11T15:56:12.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:12 smithi134 ceph-mon[130447]: pgmap v301: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 60 KiB/s wr, 20 op/s 2023-08-11T15:56:13.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:13 smithi032 ceph-mon[97103]: pgmap v302: 129 pgs: 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 109 active+clean; 62 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:13.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:13 smithi192 ceph-mon[130806]: pgmap v302: 129 pgs: 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 109 active+clean; 62 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:13.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:13 smithi134 ceph-mon[130447]: pgmap v302: 129 pgs: 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 109 active+clean; 62 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:16 smithi134 ceph-mon[130447]: pgmap v303: 129 pgs: 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 109 active+clean; 62 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:16.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:16 smithi032 ceph-mon[97103]: pgmap v303: 129 pgs: 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 109 active+clean; 62 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:16.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:16 smithi192 ceph-mon[130806]: pgmap v303: 129 pgs: 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 109 active+clean; 62 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:18 smithi192 ceph-mon[130806]: pgmap v304: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 58 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:18 smithi134 ceph-mon[130447]: pgmap v304: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 58 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:18.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:18 smithi032 ceph-mon[97103]: pgmap v304: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 58 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.2 KiB/s wr, 2 op/s 2023-08-11T15:56:20.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:20 smithi134 ceph-mon[130447]: pgmap v305: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 2 op/s 2023-08-11T15:56:20.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:20 smithi032 ceph-mon[97103]: pgmap v305: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 2 op/s 2023-08-11T15:56:20.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:20 smithi192 ceph-mon[130806]: pgmap v305: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 2 op/s 2023-08-11T15:56:22.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:22 smithi134 ceph-mon[130447]: pgmap v306: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2023-08-11T15:56:22.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:22 smithi134 ceph-mon[130447]: osdmap e107: 12 total, 12 up, 12 in 2023-08-11T15:56:22.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:22 smithi032 ceph-mon[97103]: pgmap v306: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2023-08-11T15:56:22.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:22 smithi032 ceph-mon[97103]: osdmap e107: 12 total, 12 up, 12 in 2023-08-11T15:56:22.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:22 smithi192 ceph-mon[130806]: pgmap v306: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2023-08-11T15:56:22.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:22 smithi192 ceph-mon[130806]: osdmap e107: 12 total, 12 up, 12 in 2023-08-11T15:56:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:24 smithi134 ceph-mon[130447]: pgmap v308: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:24 smithi032 ceph-mon[97103]: pgmap v308: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:24.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:24 smithi192 ceph-mon[130806]: pgmap v308: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:26 smithi134 ceph-mon[130447]: pgmap v309: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:26.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:26 smithi032 ceph-mon[97103]: pgmap v309: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:26.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:26 smithi192 ceph-mon[130806]: pgmap v309: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:28.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:28 smithi192 ceph-mon[130806]: pgmap v310: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:28.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:28 smithi134 ceph-mon[130447]: pgmap v310: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:28.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:28 smithi032 ceph-mon[97103]: pgmap v310: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:30 smithi134 ceph-mon[130447]: pgmap v311: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:30.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:30 smithi032 ceph-mon[97103]: pgmap v311: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:30.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:30 smithi192 ceph-mon[130806]: pgmap v311: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:32.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:32 smithi134 ceph-mon[130447]: pgmap v312: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:32.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:32 smithi032 ceph-mon[97103]: pgmap v312: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:32.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:32 smithi192 ceph-mon[130806]: pgmap v312: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:56:34.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:34 smithi134 ceph-mon[130447]: pgmap v313: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-08-11T15:56:34.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:34 smithi032 ceph-mon[97103]: pgmap v313: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-08-11T15:56:34.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:34 smithi192 ceph-mon[130806]: pgmap v313: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-08-11T15:56:36.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:36 smithi134 ceph-mon[130447]: pgmap v314: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:36.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:36 smithi032 ceph-mon[97103]: pgmap v314: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:36.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:36 smithi192 ceph-mon[130806]: pgmap v314: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:38.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:38 smithi192 ceph-mon[130806]: pgmap v315: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:56:38.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:38 smithi134 ceph-mon[130447]: pgmap v315: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:56:38.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:38 smithi032 ceph-mon[97103]: pgmap v315: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:56:40.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:40 smithi134 ceph-mon[130447]: pgmap v316: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-08-11T15:56:40.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:40 smithi032 ceph-mon[97103]: pgmap v316: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-08-11T15:56:40.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:40 smithi192 ceph-mon[130806]: pgmap v316: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-08-11T15:56:42.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:42 smithi134 ceph-mon[130447]: pgmap v317: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:42.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:42 smithi032 ceph-mon[97103]: pgmap v317: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:42.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:42 smithi192 ceph-mon[130806]: pgmap v317: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:43.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-08-11T15:56:44.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 -- 172.21.15.32:0/1666844752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 msgr2=0x7f4860100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 --2- 172.21.15.32:0/1666844752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860100d30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4850009990 tx=0x7f4850009ca0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 -- 172.21.15.32:0/1666844752 shutdown_connections 2023-08-11T15:56:44.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 --2- 172.21.15.32:0/1666844752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4860101270 0x7f486010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 --2- 172.21.15.32:0/1666844752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48601068c0 0x7f4860106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 --2- 172.21.15.32:0/1666844752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860100d30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 -- 172.21.15.32:0/1666844752 >> 172.21.15.32:0/1666844752 conn(0x7f48600fc380 msgr2=0x7f48600fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:44.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 -- 172.21.15.32:0/1666844752 shutdown_connections 2023-08-11T15:56:44.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.050+0000 7f4867dff700 1 -- 172.21.15.32:0/1666844752 wait complete. 2023-08-11T15:56:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.051+0000 7f4867dff700 1 Processor -- start 2023-08-11T15:56:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.051+0000 7f4867dff700 1 -- start start 2023-08-11T15:56:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4867dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4867dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4860101270 0x7f4860073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4867dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48601068c0 0x7f48601a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4867dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a50f0 con 0x7f48601008b0 2023-08-11T15:56:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4867dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a5230 con 0x7f48601068c0 2023-08-11T15:56:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4867dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a53a0 con 0x7f4860101270 2023-08-11T15:56:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4865b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4865b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34160/0 (socket says 172.21.15.32:34160) 2023-08-11T15:56:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4865b9b700 1 -- 172.21.15.32:0/1325522018 learned_addr learned my addr 172.21.15.32:0/1325522018 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f486639c700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48601068c0 0x7f48601a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f486539a700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4860101270 0x7f4860073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.052+0000 7f4865b9b700 1 -- 172.21.15.32:0/1325522018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4860101270 msgr2=0x7f4860073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4865b9b700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4860101270 0x7f4860073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4865b9b700 1 -- 172.21.15.32:0/1325522018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48601068c0 msgr2=0x7f48601a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4865b9b700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48601068c0 0x7f48601a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4865b9b700 1 -- 172.21.15.32:0/1325522018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48500095f0 con 0x7f48601008b0 2023-08-11T15:56:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4865b9b700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860072e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f485c00e9a0 tx=0x7f485c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4856ffd700 1 -- 172.21.15.32:0/1325522018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c009790 con 0x7f48601008b0 2023-08-11T15:56:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4856ffd700 1 -- 172.21.15.32:0/1325522018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f485c00cbd0 con 0x7f48601008b0 2023-08-11T15:56:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48601a5680 con 0x7f48601008b0 2023-08-11T15:56:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4856ffd700 1 -- 172.21.15.32:0/1325522018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c018680 con 0x7f48601008b0 2023-08-11T15:56:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.053+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48601a5b90 con 0x7f48601008b0 2023-08-11T15:56:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.054+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48600ffff0 con 0x7f48601008b0 2023-08-11T15:56:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.056+0000 7f4856ffd700 1 -- 172.21.15.32:0/1325522018 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f485c00cd40 con 0x7f48601008b0 2023-08-11T15:56:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.059+0000 7f4856ffd700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f484c0a00c0 0x7f484c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.059+0000 7f4856ffd700 1 -- 172.21.15.32:0/1325522018 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f485c014070 con 0x7f48601008b0 2023-08-11T15:56:44.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.059+0000 7f486539a700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f484c0a00c0 0x7f484c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.060+0000 7f486539a700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f484c0a00c0 0x7f484c0a2580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f485000b2e0 tx=0x7f4850005fb0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:44.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.062+0000 7f4856ffd700 1 -- 172.21.15.32:0/1325522018 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f485c0955a0 con 0x7f48601008b0 2023-08-11T15:56:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.297+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 --> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f4860110880 con 0x7f484c0a00c0 2023-08-11T15:56:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.307+0000 7f4856ffd700 1 -- 172.21.15.32:0/1325522018 <== mgr.15315 v2:172.21.15.32:6800/1148710426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f4860110880 con 0x7f484c0a00c0 2023-08-11T15:56:44.310 INFO:teuthology.orchestra.run.smithi032.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-08-11T00:00:00", "created": "2023-08-11T15:46:42", "first": "2023-08-11T15:47:00", "last": "2023-08-11T15:56:00", "last_pruned": "2023-08-11T15:56:01", "created_count": 10, "pruned_count": 4, "active": true} 2023-08-11T15:56:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f484c0a00c0 msgr2=0x7f484c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f484c0a00c0 0x7f484c0a2580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f485000b2e0 tx=0x7f4850005fb0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 msgr2=0x7f4860072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860072e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f485c00e9a0 tx=0x7f485c00ed60 comp rx=0 tx=0).stop 2023-08-11T15:56:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 shutdown_connections 2023-08-11T15:56:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f484c0a00c0 0x7f484c0a2580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48601008b0 0x7f4860072e00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48601068c0 0x7f48601a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 --2- 172.21.15.32:0/1325522018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4860101270 0x7f4860073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 >> 172.21.15.32:0/1325522018 conn(0x7f48600fc380 msgr2=0x7f48601043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 shutdown_connections 2023-08-11T15:56:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.310+0000 7f4867dff700 1 -- 172.21.15.32:0/1325522018 wait complete. 2023-08-11T15:56:44.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-08-11T15:56:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2061302522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241068c0 msgr2=0x7ff824106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2061302522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241068c0 0x7ff824106ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff820009990 tx=0x7ff820009ca0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2061302522 shutdown_connections 2023-08-11T15:56:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2061302522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241068c0 0x7ff824106ca0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2061302522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff824101270 0x7ff82410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2061302522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8241008b0 0x7ff824100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2061302522 >> 172.21.15.32:0/2061302522 conn(0x7ff8240fc380 msgr2=0x7ff8240fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2061302522 shutdown_connections 2023-08-11T15:56:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.530+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2061302522 wait complete. 2023-08-11T15:56:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.531+0000 7ff82c5ff700 1 Processor -- start 2023-08-11T15:56:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82c5ff700 1 -- start start 2023-08-11T15:56:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82c5ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241008b0 0x7ff824072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82c5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff824101270 0x7ff824073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82c5ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8241068c0 0x7ff8241a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82c5ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a5060 con 0x7ff8241008b0 2023-08-11T15:56:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82c5ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a51a0 con 0x7ff8241068c0 2023-08-11T15:56:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82c5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a5310 con 0x7ff824101270 2023-08-11T15:56:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82a39b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241008b0 0x7ff824072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82a39b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241008b0 0x7ff824072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34168/0 (socket says 172.21.15.32:34168) 2023-08-11T15:56:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82a39b700 1 -- 172.21.15.32:0/2121729560 learned_addr learned my addr 172.21.15.32:0/2121729560 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff82ab9c700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8241068c0 0x7ff8241a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.532+0000 7ff829b9a700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff824101270 0x7ff824073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82a39b700 1 -- 172.21.15.32:0/2121729560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff824101270 msgr2=0x7ff824073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82a39b700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff824101270 0x7ff824073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82a39b700 1 -- 172.21.15.32:0/2121729560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8241068c0 msgr2=0x7ff8241a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82a39b700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8241068c0 0x7ff8241a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82a39b700 1 -- 172.21.15.32:0/2121729560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200095f0 con 0x7ff8241008b0 2023-08-11T15:56:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82a39b700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241008b0 0x7ff824072e00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff820000c00 tx=0x7ff820004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff81b7fe700 1 -- 172.21.15.32:0/2121729560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82001d070 con 0x7ff8241008b0 2023-08-11T15:56:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff81b7fe700 1 -- 172.21.15.32:0/2121729560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff820021410 con 0x7ff8241008b0 2023-08-11T15:56:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff81b7fe700 1 -- 172.21.15.32:0/2121729560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820017650 con 0x7ff8241008b0 2023-08-11T15:56:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8241a55f0 con 0x7ff8241008b0 2023-08-11T15:56:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.533+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8241a5ab0 con 0x7ff8241008b0 2023-08-11T15:56:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.534+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8240ffff0 con 0x7ff8241008b0 2023-08-11T15:56:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.537+0000 7ff81b7fe700 1 -- 172.21.15.32:0/2121729560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7ff820021580 con 0x7ff8241008b0 2023-08-11T15:56:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.538+0000 7ff81b7fe700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff8100a00c0 0x7ff8100a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.538+0000 7ff81b7fe700 1 -- 172.21.15.32:0/2121729560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7ff8200cd3e0 con 0x7ff8241008b0 2023-08-11T15:56:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.538+0000 7ff829b9a700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff8100a00c0 0x7ff8100a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.539+0000 7ff829b9a700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff8100a00c0 0x7ff8100a2580 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff824074620 tx=0x7ff814009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.540+0000 7ff81b7fe700 1 -- 172.21.15.32:0/2121729560 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff820096af0 con 0x7ff8241008b0 2023-08-11T15:56:44.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:44 smithi134 ceph-mon[130447]: pgmap v318: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:56:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.775+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 --> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff8241107d0 con 0x7ff8100a00c0 2023-08-11T15:56:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.785+0000 7ff81b7fe700 1 -- 172.21.15.32:0/2121729560 <== mgr.15315 v2:172.21.15.32:6800/1148710426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ff8241107d0 con 0x7ff8100a00c0 2023-08-11T15:56:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:/ 1M 6M3h 2023-08-11T15:56:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff8100a00c0 msgr2=0x7ff8100a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff8100a00c0 0x7ff8100a2580 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff824074620 tx=0x7ff814009260 comp rx=0 tx=0).stop 2023-08-11T15:56:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241008b0 msgr2=0x7ff824072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241008b0 0x7ff824072e00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff820000c00 tx=0x7ff820004790 comp rx=0 tx=0).stop 2023-08-11T15:56:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 shutdown_connections 2023-08-11T15:56:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff8100a00c0 0x7ff8100a2580 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8241008b0 0x7ff824072e00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8241068c0 0x7ff8241a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 --2- 172.21.15.32:0/2121729560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff824101270 0x7ff824073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:44.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 >> 172.21.15.32:0/2121729560 conn(0x7ff8240fc380 msgr2=0x7ff8240fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:44.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 shutdown_connections 2023-08-11T15:56:44.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:44.787+0000 7ff82c5ff700 1 -- 172.21.15.32:0/2121729560 wait complete. 2023-08-11T15:56:44.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:44 smithi032 ceph-mon[97103]: pgmap v318: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:56:44.808 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-08-11T15:56:44.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:56:44.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:44 smithi192 ceph-mon[130806]: pgmap v318: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-08-11T15:56:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.020+0000 7f46631ff700 1 -- 172.21.15.32:0/1284190845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c072d50 msgr2=0x7f465c073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.020+0000 7f46631ff700 1 --2- 172.21.15.32:0/1284190845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c072d50 0x7f465c073130 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4650009990 tx=0x7f4650009ca0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.021+0000 7f46631ff700 1 -- 172.21.15.32:0/1284190845 shutdown_connections 2023-08-11T15:56:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.021+0000 7f46631ff700 1 --2- 172.21.15.32:0/1284190845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c072d50 0x7f465c073130 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.021+0000 7f46631ff700 1 --2- 172.21.15.32:0/1284190845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465c10d0d0 0x7f465c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.021+0000 7f46631ff700 1 --2- 172.21.15.32:0/1284190845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465c073670 0x7f465c10cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.021+0000 7f46631ff700 1 -- 172.21.15.32:0/1284190845 >> 172.21.15.32:0/1284190845 conn(0x7f465c0fc340 msgr2=0x7f465c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.021+0000 7f46631ff700 1 -- 172.21.15.32:0/1284190845 shutdown_connections 2023-08-11T15:56:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.021+0000 7f46631ff700 1 -- 172.21.15.32:0/1284190845 wait complete. 2023-08-11T15:56:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 Processor -- start 2023-08-11T15:56:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 -- start start 2023-08-11T15:56:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465c072d50 0x7f465c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c073670 0x7f465c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465c10d0d0 0x7f465c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1990a0 con 0x7f465c073670 2023-08-11T15:56:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c199210 con 0x7f465c072d50 2023-08-11T15:56:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f46631ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1993d0 con 0x7f465c10d0d0 2023-08-11T15:56:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f465bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c073670 0x7f465c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.022+0000 7f465bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c073670 0x7f465c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47756/0 (socket says 172.21.15.32:47756) 2023-08-11T15:56:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f465bfff700 1 -- 172.21.15.32:0/1995701682 learned_addr learned my addr 172.21.15.32:0/1995701682 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f466179c700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465c10d0d0 0x7f465c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f4660f9b700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465c072d50 0x7f465c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f465bfff700 1 -- 172.21.15.32:0/1995701682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465c10d0d0 msgr2=0x7f465c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f465bfff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465c10d0d0 0x7f465c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f465bfff700 1 -- 172.21.15.32:0/1995701682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465c072d50 msgr2=0x7f465c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f465bfff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465c072d50 0x7f465c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f465bfff700 1 -- 172.21.15.32:0/1995701682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500095f0 con 0x7f465c073670 2023-08-11T15:56:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f466179c700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465c10d0d0 0x7f465c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:56:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f4660f9b700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465c072d50 0x7f465c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:56:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f465bfff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c073670 0x7f465c19ef00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f464800e9a0 tx=0x7f464800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f4659ffb700 1 -- 172.21.15.32:0/1995701682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648009790 con 0x7f465c073670 2023-08-11T15:56:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f4659ffb700 1 -- 172.21.15.32:0/1995701682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464800cbd0 con 0x7f465c073670 2023-08-11T15:56:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f4659ffb700 1 -- 172.21.15.32:0/1995701682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648018680 con 0x7f465c073670 2023-08-11T15:56:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.023+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c1996b0 con 0x7f465c073670 2023-08-11T15:56:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.024+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c1a59b0 con 0x7f465c073670 2023-08-11T15:56:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.025+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c067710 con 0x7f465c073670 2023-08-11T15:56:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.026+0000 7f4659ffb700 1 -- 172.21.15.32:0/1995701682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f464800cd40 con 0x7f465c073670 2023-08-11T15:56:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.029+0000 7f4659ffb700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f464c0a0180 0x7f464c0a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.029+0000 7f4659ffb700 1 -- 172.21.15.32:0/1995701682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f4648014070 con 0x7f465c073670 2023-08-11T15:56:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.029+0000 7f4660f9b700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f464c0a0180 0x7f464c0a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.029+0000 7f4660f9b700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f464c0a0180 0x7f464c0a2640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f465000b2e0 tx=0x7f4650005c00 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.031+0000 7f4659ffb700 1 -- 172.21.15.32:0/1995701682 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4648051c30 con 0x7f465c073670 2023-08-11T15:56:45.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.243+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f465c112c80 con 0x7f465c073670 2023-08-11T15:56:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.244+0000 7f4659ffb700 1 -- 172.21.15.32:0/1995701682 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25072 (secure 0 0 0) 0x7f4648098e10 con 0x7f465c073670 2023-08-11T15:56:45.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:56:45.248 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":107,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:56:22.048097+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2323942786":"2023-08-12T15:46:41.134034+0000","172.21.15.32:6801/2482310260":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/1349164373":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/2111930192":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/2001360921":"2023-08-12T15:46:41.134034+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/2482310260":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:56:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f464c0a0180 msgr2=0x7f464c0a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f464c0a0180 0x7f464c0a2640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f465000b2e0 tx=0x7f4650005c00 comp rx=0 tx=0).stop 2023-08-11T15:56:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c073670 msgr2=0x7f465c19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c073670 0x7f465c19ef00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f464800e9a0 tx=0x7f464800ed60 comp rx=0 tx=0).stop 2023-08-11T15:56:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 shutdown_connections 2023-08-11T15:56:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f464c0a0180 0x7f464c0a2640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465c10d0d0 0x7f465c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465c072d50 0x7f465c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 --2- 172.21.15.32:0/1995701682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f465c073670 0x7f465c19ef00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 >> 172.21.15.32:0/1995701682 conn(0x7f465c0fc340 msgr2=0x7f465c10f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 shutdown_connections 2023-08-11T15:56:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.247+0000 7f46631ff700 1 -- 172.21.15.32:0/1995701682 wait complete. 2023-08-11T15:56:45.268 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:56:45.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:56:45.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 -- 172.21.15.32:0/2364842779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 msgr2=0x7f416010a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 --2- 172.21.15.32:0/2364842779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f416010a830 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4150009990 tx=0x7f4150009ca0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 -- 172.21.15.32:0/2364842779 shutdown_connections 2023-08-11T15:56:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 --2- 172.21.15.32:0/2364842779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f416010ae50 0x7f416010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 --2- 172.21.15.32:0/2364842779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160101a30 0x7f4160101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 --2- 172.21.15.32:0/2364842779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f416010a830 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 -- 172.21.15.32:0/2364842779 >> 172.21.15.32:0/2364842779 conn(0x7f41600fb270 msgr2=0x7f41600fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.476+0000 7f4167bff700 1 -- 172.21.15.32:0/2364842779 shutdown_connections 2023-08-11T15:56:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.477+0000 7f4167bff700 1 -- 172.21.15.32:0/2364842779 wait complete. 2023-08-11T15:56:45.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 Processor -- start 2023-08-11T15:56:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 -- start start 2023-08-11T15:56:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160101a30 0x7f41600ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f41600ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f416010ae50 0x7f41601a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41601003f0 con 0x7f4160102350 2023-08-11T15:56:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41601a5130 con 0x7f4160101a30 2023-08-11T15:56:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f4167bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41601a5270 con 0x7f416010ae50 2023-08-11T15:56:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f416519a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f41600ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f416519a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f41600ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47782/0 (socket says 172.21.15.32:47782) 2023-08-11T15:56:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.478+0000 7f416519a700 1 -- 172.21.15.32:0/3029754838 learned_addr learned my addr 172.21.15.32:0/3029754838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416599b700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160101a30 0x7f41600ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416619c700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f416010ae50 0x7f41601a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416519a700 1 -- 172.21.15.32:0/3029754838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f416010ae50 msgr2=0x7f41601a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416519a700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f416010ae50 0x7f41601a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416519a700 1 -- 172.21.15.32:0/3029754838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160101a30 msgr2=0x7f41600ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416519a700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160101a30 0x7f41600ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416519a700 1 -- 172.21.15.32:0/3029754838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500095f0 con 0x7f4160102350 2023-08-11T15:56:45.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416619c700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f416010ae50 0x7f41601a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T15:56:45.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f416519a700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f41600ffbc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4150005910 tx=0x7f4150005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:45.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f4156ffd700 1 -- 172.21.15.32:0/3029754838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415001d070 con 0x7f4160102350 2023-08-11T15:56:45.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f4156ffd700 1 -- 172.21.15.32:0/3029754838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4150022470 con 0x7f4160102350 2023-08-11T15:56:45.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f4156ffd700 1 -- 172.21.15.32:0/3029754838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150017650 con 0x7f4160102350 2023-08-11T15:56:45.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.479+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41601a5470 con 0x7f4160102350 2023-08-11T15:56:45.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.480+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41601a58f0 con 0x7f4160102350 2023-08-11T15:56:45.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.481+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4160105f20 con 0x7f4160102350 2023-08-11T15:56:45.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.484+0000 7f4156ffd700 1 -- 172.21.15.32:0/3029754838 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f41500225e0 con 0x7f4160102350 2023-08-11T15:56:45.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.484+0000 7f4156ffd700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f414c0a0140 0x7f414c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.484+0000 7f4156ffd700 1 -- 172.21.15.32:0/3029754838 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f41500cccb0 con 0x7f4160102350 2023-08-11T15:56:45.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.484+0000 7f416599b700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f414c0a0140 0x7f414c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.485+0000 7f416599b700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f414c0a0140 0x7f414c0a2600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f415c009c60 tx=0x7f415c009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:45.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.487+0000 7f4156ffd700 1 -- 172.21.15.32:0/3029754838 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f41500963c0 con 0x7f4160102350 2023-08-11T15:56:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:45 smithi032 ceph-mon[97103]: from='client.16056 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:56:45.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:45 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1995701682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:56:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.699+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41601108c0 con 0x7f4160102350 2023-08-11T15:56:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.699+0000 7f4156ffd700 1 -- 172.21.15.32:0/3029754838 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41500999e0 con 0x7f4160102350 2023-08-11T15:56:45.702 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:56:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f414c0a0140 msgr2=0x7f414c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f414c0a0140 0x7f414c0a2600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f415c009c60 tx=0x7f415c009260 comp rx=0 tx=0).stop 2023-08-11T15:56:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 msgr2=0x7f41600ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f41600ffbc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4150005910 tx=0x7f4150005740 comp rx=0 tx=0).stop 2023-08-11T15:56:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 shutdown_connections 2023-08-11T15:56:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f414c0a0140 0x7f414c0a2600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f416010ae50 0x7f41601a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4160101a30 0x7f41600ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 --2- 172.21.15.32:0/3029754838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4160102350 0x7f41600ffbc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 >> 172.21.15.32:0/3029754838 conn(0x7f41600fb270 msgr2=0x7f41600fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 shutdown_connections 2023-08-11T15:56:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.702+0000 7f4167bff700 1 -- 172.21.15.32:0/3029754838 wait complete. 2023-08-11T15:56:45.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:45 smithi134 ceph-mon[130447]: from='client.16056 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:56:45.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:45 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1995701682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:56:45.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:56:45.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:45 smithi192 ceph-mon[130806]: from='client.16056 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:56:45.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:45 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1995701682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:56:45.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 -- 172.21.15.32:0/2816276886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 msgr2=0x7f2074106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 --2- 172.21.15.32:0/2816276886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 0x7f2074106c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2070009990 tx=0x7f2070009ca0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 -- 172.21.15.32:0/2816276886 shutdown_connections 2023-08-11T15:56:45.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 --2- 172.21.15.32:0/2816276886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 0x7f2074106c80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 --2- 172.21.15.32:0/2816276886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074101250 0x7f207410d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 --2- 172.21.15.32:0/2816276886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2074100890 0x7f2074100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 -- 172.21.15.32:0/2816276886 >> 172.21.15.32:0/2816276886 conn(0x7f20740fc380 msgr2=0x7f20740fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:45.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 -- 172.21.15.32:0/2816276886 shutdown_connections 2023-08-11T15:56:45.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.929+0000 7f207bdff700 1 -- 172.21.15.32:0/2816276886 wait complete. 2023-08-11T15:56:45.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.930+0000 7f207bdff700 1 Processor -- start 2023-08-11T15:56:45.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.930+0000 7f207bdff700 1 -- start start 2023-08-11T15:56:45.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207bdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074100890 0x7f207419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207bdff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2074101250 0x7f207419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207bdff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 0x7f20741989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207bdff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2074199170 con 0x7f20741068a0 2023-08-11T15:56:45.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207bdff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20741992e0 con 0x7f2074101250 2023-08-11T15:56:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207bdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20741994a0 con 0x7f2074100890 2023-08-11T15:56:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f2079b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074100890 0x7f207419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f2079b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074100890 0x7f207419e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38548/0 (socket says 172.21.15.32:38548) 2023-08-11T15:56:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f2079b9b700 1 -- 172.21.15.32:0/2749641590 learned_addr learned my addr 172.21.15.32:0/2749641590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207939a700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2074101250 0x7f207419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f207a39c700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 0x7f20741989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f2079b9b700 1 -- 172.21.15.32:0/2749641590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2074101250 msgr2=0x7f207419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.931+0000 7f2079b9b700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2074101250 0x7f207419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f2079b9b700 1 -- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 msgr2=0x7f20741989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f2079b9b700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 0x7f20741989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f2079b9b700 1 -- 172.21.15.32:0/2749641590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700095f0 con 0x7f2074100890 2023-08-11T15:56:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f2079b9b700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074100890 0x7f207419e9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2070005910 tx=0x7f2070004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f206affd700 1 -- 172.21.15.32:0/2749641590 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207001d070 con 0x7f2074100890 2023-08-11T15:56:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2074199720 con 0x7f2074100890 2023-08-11T15:56:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20741a5940 con 0x7f2074100890 2023-08-11T15:56:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f206affd700 1 -- 172.21.15.32:0/2749641590 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2070022470 con 0x7f2074100890 2023-08-11T15:56:45.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.932+0000 7f206affd700 1 -- 172.21.15.32:0/2749641590 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070017650 con 0x7f2074100890 2023-08-11T15:56:45.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.933+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2058005320 con 0x7f2074100890 2023-08-11T15:56:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.935+0000 7f206affd700 1 -- 172.21.15.32:0/2749641590 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f207000fad0 con 0x7f2074100890 2023-08-11T15:56:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.937+0000 7f206affd700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20600a00c0 0x7f20600a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.937+0000 7f206affd700 1 -- 172.21.15.32:0/2749641590 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f20700ccc80 con 0x7f2074100890 2023-08-11T15:56:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.937+0000 7f207939a700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20600a00c0 0x7f20600a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.937+0000 7f207939a700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20600a00c0 0x7f20600a2580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f207419a1d0 tx=0x7f20640073d0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:45.939+0000 7f206affd700 1 -- 172.21.15.32:0/2749641590 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2070096390 con 0x7f2074100890 2023-08-11T15:56:46.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.151+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2058005b50 con 0x7f2074100890 2023-08-11T15:56:46.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.151+0000 7f206affd700 1 -- 172.21.15.32:0/2749641590 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20700999b0 con 0x7f2074100890 2023-08-11T15:56:46.154 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:56:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20600a00c0 msgr2=0x7f20600a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20600a00c0 0x7f20600a2580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f207419a1d0 tx=0x7f20640073d0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074100890 msgr2=0x7f207419e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074100890 0x7f207419e9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2070005910 tx=0x7f2070004730 comp rx=0 tx=0).stop 2023-08-11T15:56:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 shutdown_connections 2023-08-11T15:56:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20600a00c0 0x7f20600a2580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20741068a0 0x7f20741989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2074100890 0x7f207419e9c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 --2- 172.21.15.32:0/2749641590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2074101250 0x7f207419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 >> 172.21.15.32:0/2749641590 conn(0x7f20740fc380 msgr2=0x7f207410bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 shutdown_connections 2023-08-11T15:56:46.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.154+0000 7f207bdff700 1 -- 172.21.15.32:0/2749641590 wait complete. 2023-08-11T15:56:46.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:56:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.381+0000 7f46f3bff700 1 -- 172.21.15.32:0/3785767340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec106880 msgr2=0x7f46ec106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.381+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3785767340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec106880 0x7f46ec106c60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009990 tx=0x7f46e8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.382+0000 7f46f3bff700 1 -- 172.21.15.32:0/3785767340 shutdown_connections 2023-08-11T15:56:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.382+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3785767340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec106880 0x7f46ec106c60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.382+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3785767340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec101230 0x7f46ec10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.382+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3785767340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 0x7f46ec100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.382+0000 7f46f3bff700 1 -- 172.21.15.32:0/3785767340 >> 172.21.15.32:0/3785767340 conn(0x7f46ec0fc3a0 msgr2=0x7f46ec0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.382+0000 7f46f3bff700 1 -- 172.21.15.32:0/3785767340 shutdown_connections 2023-08-11T15:56:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.382+0000 7f46f3bff700 1 -- 172.21.15.32:0/3785767340 wait complete. 2023-08-11T15:56:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.383+0000 7f46f3bff700 1 Processor -- start 2023-08-11T15:56:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.383+0000 7f46f3bff700 1 -- start start 2023-08-11T15:56:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f3bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 0x7f46ec072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f3bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec101230 0x7f46ec073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f3bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec106880 0x7f46ec1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f3bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec073990 con 0x7f46ec101230 2023-08-11T15:56:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f3bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec1a5040 con 0x7f46ec106880 2023-08-11T15:56:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f3bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec1a5180 con 0x7f46ec100870 2023-08-11T15:56:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f119a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec101230 0x7f46ec073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f199b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 0x7f46ec072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f199b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 0x7f46ec072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:38566/0 (socket says 172.21.15.32:38566) 2023-08-11T15:56:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.384+0000 7f46f199b700 1 -- 172.21.15.32:0/3916601098 learned_addr learned my addr 172.21.15.32:0/3916601098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f219c700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec106880 0x7f46ec1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f199b700 1 -- 172.21.15.32:0/3916601098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec106880 msgr2=0x7f46ec1a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f199b700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec106880 0x7f46ec1a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f199b700 1 -- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec101230 msgr2=0x7f46ec073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f199b700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec101230 0x7f46ec073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f199b700 1 -- 172.21.15.32:0/3916601098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80095f0 con 0x7f46ec100870 2023-08-11T15:56:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f199b700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 0x7f46ec072c20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b2e0 tx=0x7f46e8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46e2ffd700 1 -- 172.21.15.32:0/3916601098 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e801d070 con 0x7f46ec100870 2023-08-11T15:56:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec1a5380 con 0x7f46ec100870 2023-08-11T15:56:46.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46e2ffd700 1 -- 172.21.15.32:0/3916601098 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46e8021410 con 0x7f46ec100870 2023-08-11T15:56:46.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46e2ffd700 1 -- 172.21.15.32:0/3916601098 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8017650 con 0x7f46ec100870 2023-08-11T15:56:46.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.385+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec1a58b0 con 0x7f46ec100870 2023-08-11T15:56:46.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.386+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec0676a0 con 0x7f46ec100870 2023-08-11T15:56:46.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.389+0000 7f46e2ffd700 1 -- 172.21.15.32:0/3916601098 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f46e8021810 con 0x7f46ec100870 2023-08-11T15:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.390+0000 7f46e2ffd700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f46d80a00c0 0x7f46d80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.390+0000 7f46e2ffd700 1 -- 172.21.15.32:0/3916601098 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f46e80cd500 con 0x7f46ec100870 2023-08-11T15:56:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.390+0000 7f46f119a700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f46d80a00c0 0x7f46d80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.391+0000 7f46f119a700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f46d80a00c0 0x7f46d80a2580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46ec074410 tx=0x7f46dc009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.391+0000 7f46e2ffd700 1 -- 172.21.15.32:0/3916601098 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f46e8096c70 con 0x7f46ec100870 2023-08-11T15:56:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:46 smithi032 ceph-mon[97103]: from='client.16062 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:56:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:46 smithi032 ceph-mon[97103]: pgmap v319: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:46.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:46 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3029754838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:56:46.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:46 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/2749641590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.603+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46ec1107a0 con 0x7f46ec100870 2023-08-11T15:56:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.604+0000 7f46e2ffd700 1 -- 172.21.15.32:0/3916601098 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46e809a290 con 0x7f46ec100870 2023-08-11T15:56:46.606 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:56:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.606+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f46d80a00c0 msgr2=0x7f46d80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.606+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f46d80a00c0 0x7f46d80a2580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46ec074410 tx=0x7f46dc009260 comp rx=0 tx=0).stop 2023-08-11T15:56:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.606+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 msgr2=0x7f46ec072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.606+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 0x7f46ec072c20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b2e0 tx=0x7f46e8004730 comp rx=0 tx=0).stop 2023-08-11T15:56:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 shutdown_connections 2023-08-11T15:56:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f46d80a00c0 0x7f46d80a2580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec106880 0x7f46ec1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46ec100870 0x7f46ec072c20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 --2- 172.21.15.32:0/3916601098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ec101230 0x7f46ec073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 >> 172.21.15.32:0/3916601098 conn(0x7f46ec0fc3a0 msgr2=0x7f46ec0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 shutdown_connections 2023-08-11T15:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.607+0000 7f46f3bff700 1 -- 172.21.15.32:0/3916601098 wait complete. 2023-08-11T15:56:46.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:56:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:46 smithi134 ceph-mon[130447]: from='client.16062 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:56:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:46 smithi134 ceph-mon[130447]: pgmap v319: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:46.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:46 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3029754838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:56:46.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:46 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/2749641590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:56:46.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:46 smithi192 ceph-mon[130806]: from='client.16062 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T15:56:46.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:46 smithi192 ceph-mon[130806]: pgmap v319: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:56:46.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:46 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3029754838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:56:46.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:46 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/2749641590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:56:46.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 -- 172.21.15.32:0/341718963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 msgr2=0x7fa86c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 --2- 172.21.15.32:0/341718963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c102f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa8600098f0 tx=0x7fa860009c00 comp rx=0 tx=0).stop 2023-08-11T15:56:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 -- 172.21.15.32:0/341718963 shutdown_connections 2023-08-11T15:56:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 --2- 172.21.15.32:0/341718963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c103460 0x7fa86c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 --2- 172.21.15.32:0/341718963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c108ab0 0x7fa86c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 --2- 172.21.15.32:0/341718963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c102f20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 -- 172.21.15.32:0/341718963 >> 172.21.15.32:0/341718963 conn(0x7fa86c0fe590 msgr2=0x7fa86c1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 -- 172.21.15.32:0/341718963 shutdown_connections 2023-08-11T15:56:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.832+0000 7fa8727ff700 1 -- 172.21.15.32:0/341718963 wait complete. 2023-08-11T15:56:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.833+0000 7fa8727ff700 1 Processor -- start 2023-08-11T15:56:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.833+0000 7fa8727ff700 1 -- start start 2023-08-11T15:56:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa8727ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa8727ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c103460 0x7fa86c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa8727ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c108ab0 0x7fa86c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa8727ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1990f0 con 0x7fa86c102aa0 2023-08-11T15:56:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa8727ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c199260 con 0x7fa86c103460 2023-08-11T15:56:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa8727ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c199420 con 0x7fa86c108ab0 2023-08-11T15:56:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa870d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c108ab0 0x7fa86c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa86b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c103460 0x7fa86c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47842/0 (socket says 172.21.15.32:47842) 2023-08-11T15:56:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.834+0000 7fa86bfff700 1 -- 172.21.15.32:0/3444855198 learned_addr learned my addr 172.21.15.32:0/3444855198 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa86b7fe700 1 -- 172.21.15.32:0/3444855198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c108ab0 msgr2=0x7fa86c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa86b7fe700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c108ab0 0x7fa86c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa86b7fe700 1 -- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 msgr2=0x7fa86c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa86b7fe700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa86b7fe700 1 -- 172.21.15.32:0/3444855198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009520 con 0x7fa86c103460 2023-08-11T15:56:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa86bfff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:56:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa86b7fe700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c103460 0x7fa86c19ef50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa860009810 tx=0x7fa86000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa8697fa700 1 -- 172.21.15.32:0/3444855198 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86001c070 con 0x7fa86c103460 2023-08-11T15:56:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa8697fa700 1 -- 172.21.15.32:0/3444855198 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa86000bba0 con 0x7fa86c103460 2023-08-11T15:56:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c1996a0 con 0x7fa86c103460 2023-08-11T15:56:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.835+0000 7fa8697fa700 1 -- 172.21.15.32:0/3444855198 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860003970 con 0x7fa86c103460 2023-08-11T15:56:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.836+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c1a5a00 con 0x7fa86c103460 2023-08-11T15:56:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.837+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c067af0 con 0x7fa86c103460 2023-08-11T15:56:46.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.839+0000 7fa8697fa700 1 -- 172.21.15.32:0/3444855198 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa860020030 con 0x7fa86c103460 2023-08-11T15:56:46.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.840+0000 7fa8697fa700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa8540a0140 0x7fa8540a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:46.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.840+0000 7fa8697fa700 1 -- 172.21.15.32:0/3444855198 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fa8600cca00 con 0x7fa86c103460 2023-08-11T15:56:46.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.840+0000 7fa86bfff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa8540a0140 0x7fa8540a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:46.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.841+0000 7fa86bfff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa8540a0140 0x7fa8540a2600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa85c005950 tx=0x7fa85c009310 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:46.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:46.842+0000 7fa8697fa700 1 -- 172.21.15.32:0/3444855198 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa860096110 con 0x7fa86c103460 2023-08-11T15:56:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.052+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa86c112b20 con 0x7fa86c103460 2023-08-11T15:56:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.053+0000 7fa8697fa700 1 -- 172.21.15.32:0/3444855198 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa86009de60 con 0x7fa86c103460 2023-08-11T15:56:47.054 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:56:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa8540a0140 msgr2=0x7fa8540a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa8540a0140 0x7fa8540a2600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa85c005950 tx=0x7fa85c009310 comp rx=0 tx=0).stop 2023-08-11T15:56:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c103460 msgr2=0x7fa86c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c103460 0x7fa86c19ef50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa860009810 tx=0x7fa86000fa00 comp rx=0 tx=0).stop 2023-08-11T15:56:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 shutdown_connections 2023-08-11T15:56:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa8540a0140 0x7fa8540a2600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa86c102aa0 0x7fa86c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c108ab0 0x7fa86c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 --2- 172.21.15.32:0/3444855198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c103460 0x7fa86c19ef50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 >> 172.21.15.32:0/3444855198 conn(0x7fa86c0fe590 msgr2=0x7fa86c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:47.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 shutdown_connections 2023-08-11T15:56:47.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.055+0000 7fa8727ff700 1 -- 172.21.15.32:0/3444855198 wait complete. 2023-08-11T15:56:47.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-08-11T15:56:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.280+0000 7f90ffdff700 1 -- 172.21.15.32:0/357676109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f8101250 msgr2=0x7f90f810d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.280+0000 7f90ffdff700 1 --2- 172.21.15.32:0/357676109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f8101250 0x7f90f810d760 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009f90 tx=0x7f90f000b330 comp rx=0 tx=0).stop 2023-08-11T15:56:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.281+0000 7f90ffdff700 1 -- 172.21.15.32:0/357676109 shutdown_connections 2023-08-11T15:56:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.281+0000 7f90ffdff700 1 --2- 172.21.15.32:0/357676109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f8101250 0x7f90f810d760 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.281+0000 7f90ffdff700 1 --2- 172.21.15.32:0/357676109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90f81068a0 0x7f90f8106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.281+0000 7f90ffdff700 1 --2- 172.21.15.32:0/357676109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f8100890 0x7f90f8100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.281+0000 7f90ffdff700 1 -- 172.21.15.32:0/357676109 >> 172.21.15.32:0/357676109 conn(0x7f90f80fc340 msgr2=0x7f90f80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.281+0000 7f90ffdff700 1 -- 172.21.15.32:0/357676109 shutdown_connections 2023-08-11T15:56:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.281+0000 7f90ffdff700 1 -- 172.21.15.32:0/357676109 wait complete. 2023-08-11T15:56:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 Processor -- start 2023-08-11T15:56:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 -- start start 2023-08-11T15:56:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f8100890 0x7f90f819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90f8101250 0x7f90f819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f81068a0 0x7f90f8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8198f90 con 0x7f90f81068a0 2023-08-11T15:56:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8199100 con 0x7f90f8100890 2023-08-11T15:56:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.282+0000 7f90ffdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81992c0 con 0x7f90f8101250 2023-08-11T15:56:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fdb9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f8100890 0x7f90f819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fe39c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f81068a0 0x7f90f8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fdb9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f8100890 0x7f90f819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:53144/0 (socket says 172.21.15.32:53144) 2023-08-11T15:56:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fdb9b700 1 -- 172.21.15.32:0/1631103243 learned_addr learned my addr 172.21.15.32:0/1631103243 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:56:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fd39a700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90f8101250 0x7f90f819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fe39c700 1 -- 172.21.15.32:0/1631103243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90f8101250 msgr2=0x7f90f819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fe39c700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90f8101250 0x7f90f819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fe39c700 1 -- 172.21.15.32:0/1631103243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f8100890 msgr2=0x7f90f819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fe39c700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f8100890 0x7f90f819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fe39c700 1 -- 172.21.15.32:0/1631103243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0009c40 con 0x7f90f81068a0 2023-08-11T15:56:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.283+0000 7f90fe39c700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f81068a0 0x7f90f8198920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b6c0 tx=0x7f90f0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.284+0000 7f90eeffd700 1 -- 172.21.15.32:0/1631103243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000e040 con 0x7f90f81068a0 2023-08-11T15:56:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.284+0000 7f90eeffd700 1 -- 172.21.15.32:0/1631103243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90f001d470 con 0x7f90f81068a0 2023-08-11T15:56:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.284+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f8199540 con 0x7f90f81068a0 2023-08-11T15:56:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.284+0000 7f90eeffd700 1 -- 172.21.15.32:0/1631103243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f00052a0 con 0x7f90f81068a0 2023-08-11T15:56:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.284+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f81a58a0 con 0x7f90f81068a0 2023-08-11T15:56:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.285+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f8067710 con 0x7f90f81068a0 2023-08-11T15:56:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.287+0000 7f90eeffd700 1 -- 172.21.15.32:0/1631103243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f90f001d5e0 con 0x7f90f81068a0 2023-08-11T15:56:47.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.288+0000 7f90eeffd700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f90e40a0180 0x7f90e40a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:56:47.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.288+0000 7f90eeffd700 1 -- 172.21.15.32:0/1631103243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f90f00cce00 con 0x7f90f81068a0 2023-08-11T15:56:47.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.288+0000 7f90fdb9b700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f90e40a0180 0x7f90e40a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:56:47.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.289+0000 7f90fdb9b700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f90e40a0180 0x7f90e40a2640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f90f4005950 tx=0x7f90f4009310 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:56:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.292+0000 7f90eeffd700 1 -- 172.21.15.32:0/1631103243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f90f0096510 con 0x7f90f81068a0 2023-08-11T15:56:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.564+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f90f8110a40 con 0x7f90f81068a0 2023-08-11T15:56:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.575+0000 7f90eeffd700 1 -- 172.21.15.32:0/1631103243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+728386 (secure 0 0 0) 0x7f90f0099b30 con 0x7f90f81068a0 2023-08-11T15:56:47.585 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:56:47.586 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:56:47.586 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 26, 2023-08-11T15:56:47.586 INFO:teuthology.orchestra.run.smithi032.stdout: "active_gid": 15315, 2023-08-11T15:56:47.586 INFO:teuthology.orchestra.run.smithi032.stdout: "active_name": "x", 2023-08-11T15:56:47.587 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addrs": { 2023-08-11T15:56:47.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-08-11T15:56:47.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-08-11T15:56:47.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2023-08-11T15:56:47.588 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1148710426 2023-08-11T15:56:47.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-08-11T15:56:47.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2023-08-11T15:56:47.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1148710426 2023-08-11T15:56:47.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.589 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:47.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.590 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addr": "172.21.15.32:6801/1148710426", 2023-08-11T15:56:47.590 INFO:teuthology.orchestra.run.smithi032.stdout: "active_change": "2023-08-11T15:46:41.134323+0000", 2023-08-11T15:56:47.590 INFO:teuthology.orchestra.run.smithi032.stdout: "active_mgr_features": 4540138322906710015, 2023-08-11T15:56:47.590 INFO:teuthology.orchestra.run.smithi032.stdout: "available": true, 2023-08-11T15:56:47.591 INFO:teuthology.orchestra.run.smithi032.stdout: "standbys": [ 2023-08-11T15:56:47.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.591 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 25318, 2023-08-11T15:56:47.592 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "z", 2023-08-11T15:56:47.592 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540138322906710015, 2023-08-11T15:56:47.593 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-08-11T15:56:47.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.593 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-08-11T15:56:47.594 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:47.594 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:47.594 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:47.595 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:47.595 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:47.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.596 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.597 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:47.597 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.598 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.598 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.599 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-08-11T15:56:47.599 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.600 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.600 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:47.601 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:47.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.602 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.602 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.603 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.604 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.604 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.606 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.606 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.606 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.607 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.607 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.608 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.608 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.609 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:47.609 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:47.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.610 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.611 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.611 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.612 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.612 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.613 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.613 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.614 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.616 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.616 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:47.616 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.616 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.616 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.617 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.617 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.617 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.617 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.618 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.618 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.618 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.618 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.619 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.619 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.620 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:47.620 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:47.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.620 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.620 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.621 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.621 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.621 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.621 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.621 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.622 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.622 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.622 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-08-11T15:56:47.623 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-08-11T15:56:47.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.623 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.623 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.623 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.624 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.624 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.624 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.624 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-08-11T15:56:47.624 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.625 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.625 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-08-11T15:56:47.625 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-08-11T15:56:47.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.626 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.627 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-08-11T15:56:47.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.628 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-08-11T15:56:47.628 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.628 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.628 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.629 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-08-11T15:56:47.629 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-08-11T15:56:47.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.629 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.630 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.630 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.630 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.630 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.631 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-08-11T15:56:47.631 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.631 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.632 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-08-11T15:56:47.632 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-08-11T15:56:47.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.632 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.633 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.633 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.634 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.634 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-08-11T15:56:47.634 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.635 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.635 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-08-11T15:56:47.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-08-11T15:56:47.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.636 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-08-11T15:56:47.636 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.636 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-08-11T15:56:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-08-11T15:56:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-08-11T15:56:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.640 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-08-11T15:56:47.640 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.640 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.641 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-08-11T15:56:47.641 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-08-11T15:56:47.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.641 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.642 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.642 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.642 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.643 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.643 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-11T15:56:47.643 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.644 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.644 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-08-11T15:56:47.644 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-08-11T15:56:47.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.645 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.645 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.645 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.645 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.646 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.646 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-08-11T15:56:47.646 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.647 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.648 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-08-11T15:56:47.648 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:47.648 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:47.648 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:47.648 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-08-11T15:56:47.649 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-08-11T15:56:47.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.649 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.650 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.650 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.650 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.650 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-08-11T15:56:47.651 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.651 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.651 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-08-11T15:56:47.652 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-08-11T15:56:47.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.652 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.653 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-08-11T15:56:47.653 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.653 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.654 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.654 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-08-11T15:56:47.654 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T15:56:47.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.654 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.655 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-08-11T15:56:47.655 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-08-11T15:56:47.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:47.655 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-08-11T15:56:47.656 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-08-11T15:56:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-11T15:56:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T15:56:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.657 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.658 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-08-11T15:56:47.659 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-08-11T15:56:47.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:47.660 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.660 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-08-11T15:56:47.661 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:47.661 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-08-11T15:56:47.662 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.662 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-11T15:56:47.663 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.663 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.664 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-08-11T15:56:47.665 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-08-11T15:56:47.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.666 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.666 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.666 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-08-11T15:56:47.666 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.667 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.667 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.668 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-11T15:56:47.668 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-11T15:56:47.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.669 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.669 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-08-11T15:56:47.669 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-08-11T15:56:47.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:47.670 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.670 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-08-11T15:56:47.670 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-08-11T15:56:47.671 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-08-11T15:56:47.671 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.671 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-08-11T15:56:47.671 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-11T15:56:47.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.672 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.672 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-08-11T15:56:47.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-08-11T15:56:47.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.673 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.673 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.673 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-08-11T15:56:47.673 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.673 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.674 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.674 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-08-11T15:56:47.674 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T15:56:47.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.674 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.675 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-08-11T15:56:47.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-08-11T15:56:47.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:47.676 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.676 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:47.677 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-08-11T15:56:47.677 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-08-11T15:56:47.677 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.677 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-11T15:56:47.677 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T15:56:47.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.678 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.678 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:47.679 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:47.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.679 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.680 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.680 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.680 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.681 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.681 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.681 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.682 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.682 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.683 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.683 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.683 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.684 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.684 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.685 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.686 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:47.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:47.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.688 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.688 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.689 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.689 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.689 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.690 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.690 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.691 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-08-11T15:56:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-08-11T15:56:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-11T15:56:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-08-11T15:56:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-08-11T15:56:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-08-11T15:56:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-08-11T15:56:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-08-11T15:56:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-08-11T15:56:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-08-11T15:56:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-08-11T15:56:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-08-11T15:56:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-11T15:56:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-11T15:56:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-08-11T15:56:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-08-11T15:56:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-11T15:56:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-11T15:56:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-08-11T15:56:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-08-11T15:56:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-11T15:56:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-08-11T15:56:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-08-11T15:56:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-08-11T15:56:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-08-11T15:56:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-11T15:56:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-08-11T15:56:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-08-11T15:56:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-08-11T15:56:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-11T15:56:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-08-11T15:56:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-08-11T15:56:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-08-11T15:56:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-11T15:56:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-08-11T15:56:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-08-11T15:56:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-11T15:56:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-11T15:56:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-08-11T15:56:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-08-11T15:56:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-11T15:56:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-08-11T15:56:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-08-11T15:56:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-08-11T15:56:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-11T15:56:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-08-11T15:56:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-08-11T15:56:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-11T15:56:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-08-11T15:56:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-08-11T15:56:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-11T15:56:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-08-11T15:56:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-08-11T15:56:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-11T15:56:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-08-11T15:56:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-08-11T15:56:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-11T15:56:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-08-11T15:56:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-08-11T15:56:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-11T15:56:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-08-11T15:56:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-08-11T15:56:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-08-11T15:56:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-11T15:56:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-08-11T15:56:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-08-11T15:56:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-08-11T15:56:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-11T15:56:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-08-11T15:56:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-08-11T15:56:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-11T15:56:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-08-11T15:56:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-08-11T15:56:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-08-11T15:56:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-11T15:56:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-08-11T15:56:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-08-11T15:56:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-08-11T15:56:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-11T15:56:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-08-11T15:56:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-08-11T15:56:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-08-11T15:56:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-11T15:56:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-08-11T15:56:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-08-11T15:56:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-08-11T15:56:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-11T15:56:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-08-11T15:56:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-08-11T15:56:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-08-11T15:56:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-11T15:56:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-08-11T15:56:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-08-11T15:56:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-08-11T15:56:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-11T15:56:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-08-11T15:56:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-08-11T15:56:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.1", 2023-08-11T15:56:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-08-11T15:56:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-08-11T15:56:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-08-11T15:56:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-11T15:56:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-08-11T15:56:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-08-11T15:56:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-11T15:56:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-08-11T15:56:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-08-11T15:56:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-08-11T15:56:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-11T15:56:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-08-11T15:56:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-08-11T15:56:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-08-11T15:56:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-08-11T15:56:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-08-11T15:56:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-08-11T15:56:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-11T15:56:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-08-11T15:56:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-08-11T15:56:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-08-11T15:56:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-11T15:56:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-08-11T15:56:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-08-11T15:56:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-08-11T15:56:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-11T15:56:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-08-11T15:56:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-08-11T15:56:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-08-11T15:56:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-08-11T15:56:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-08-11T15:56:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-08-11T15:56:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-11T15:56:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-08-11T15:56:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-08-11T15:56:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-08-11T15:56:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-08-11T15:56:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-08-11T15:56:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-08-11T15:56:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-08-11T15:56:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-08-11T15:56:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-11T15:56:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-11T15:56:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-11T15:56:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-11T15:56:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-11T15:56:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-11T15:56:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-11T15:56:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-08-11T15:56:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-11T15:56:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-08-11T15:56:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-08-11T15:56:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-08-11T15:56:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-08-11T15:56:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-08-11T15:56:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-11T15:56:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-08-11T15:56:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-08-11T15:56:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-08-11T15:56:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-08-11T15:56:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-08-11T15:56:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-08-11T15:56:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-08-11T15:56:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-08-11T15:56:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-08-11T15:56:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-08-11T15:56:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-08-11T15:56:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-11T15:56:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-11T15:56:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-08-11T15:56:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-08-11T15:56:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-11T15:56:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-08-11T15:56:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-08-11T15:56:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-11T15:56:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-08-11T15:56:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-08-11T15:56:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-11T15:56:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-08-11T15:56:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-08-11T15:56:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-11T15:56:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-08-11T15:56:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-08-11T15:56:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-11T15:56:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-08-11T15:56:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-08-11T15:56:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-08-11T15:56:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-08-11T15:56:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.856 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.856 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-08-11T15:56:47.856 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-08-11T15:56:47.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.857 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.857 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.857 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.858 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.858 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.858 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-11T15:56:47.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.859 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.859 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-08-11T15:56:47.859 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-08-11T15:56:47.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.860 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.861 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.861 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-11T15:56:47.861 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.861 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.862 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-08-11T15:56:47.862 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-08-11T15:56:47.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.862 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.863 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.863 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.863 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.863 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-11T15:56:47.864 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.864 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.864 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-08-11T15:56:47.865 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-08-11T15:56:47.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.865 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.865 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.866 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.866 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.866 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.866 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-08-11T15:56:47.866 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.867 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.867 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-08-11T15:56:47.867 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-08-11T15:56:47.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.868 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.868 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.868 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.868 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.869 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.869 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-11T15:56:47.869 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.869 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.869 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.870 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-08-11T15:56:47.870 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-08-11T15:56:47.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.870 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.871 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.871 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.871 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.871 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.871 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.872 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-11T15:56:47.872 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.872 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.873 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-08-11T15:56:47.873 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:47.874 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:47.874 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:47.874 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:47.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:47.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.874 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.875 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.875 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.875 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.875 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.876 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.876 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.876 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.876 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.876 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.877 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.877 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.877 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.877 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.877 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.878 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.878 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:47.878 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:47.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.878 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.879 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.879 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.879 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.879 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.880 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.880 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.880 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.881 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:47.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:47.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.881 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.881 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.882 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:47.882 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:47.882 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:47.884 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:47.884 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.884 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.884 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.885 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:47.885 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:47.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.885 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.886 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.886 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.886 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.886 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.886 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.887 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.887 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.887 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-08-11T15:56:47.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-08-11T15:56:47.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.888 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.888 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-08-11T15:56:47.888 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-11T15:56:47.889 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-08-11T15:56:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-08-11T15:56:47.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-08-11T15:56:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.891 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.892 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.892 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-11T15:56:47.892 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.892 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:47.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:47.893 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-08-11T15:56:47.894 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:47.894 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:47.894 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:47.894 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-11T15:56:47.894 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-11T15:56:47.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.895 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.895 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:47.895 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.896 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.896 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.896 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.896 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.897 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.897 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-08-11T15:56:47.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-11T15:56:47.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.898 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.898 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.898 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.898 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.899 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.900 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-11T15:56:47.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-11T15:56:47.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.900 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.901 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.901 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.901 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.901 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.902 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.902 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.902 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-08-11T15:56:47.903 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-08-11T15:56:47.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.903 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.903 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.903 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.903 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.904 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.904 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.904 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.905 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-11T15:56:47.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-11T15:56:47.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.906 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.907 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.907 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.907 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.907 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.908 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-08-11T15:56:47.908 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-11T15:56:47.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.908 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.909 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.909 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.909 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.909 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.909 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.910 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.910 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.910 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-11T15:56:47.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-11T15:56:47.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.911 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.911 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.912 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.912 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.912 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.912 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.912 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.913 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.913 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-11T15:56:47.913 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-11T15:56:47.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.914 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.914 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.914 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.915 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.915 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.915 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.915 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.916 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-11T15:56:47.916 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-11T15:56:47.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.916 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.917 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.917 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.917 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.917 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.918 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.918 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.918 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.918 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-11T15:56:47.919 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-11T15:56:47.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.919 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.919 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.920 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.920 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.920 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.920 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.920 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.921 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.921 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-11T15:56:47.921 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-11T15:56:47.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.922 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.922 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.922 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.923 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.923 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.923 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.923 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.923 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.924 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-11T15:56:47.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-11T15:56:47.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.924 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.925 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.925 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.925 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.925 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.926 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.927 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.927 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-11T15:56:47.927 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-11T15:56:47.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.928 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.928 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.929 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.929 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.930 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.930 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-11T15:56:47.930 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-11T15:56:47.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.931 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.931 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.932 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.932 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.933 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.933 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-08-11T15:56:47.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-11T15:56:47.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.933 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.934 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:47.934 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.934 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.934 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.935 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.935 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.935 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.936 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-11T15:56:47.936 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-11T15:56:47.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.936 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.937 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.938 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-08-11T15:56:47.939 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-08-11T15:56:47.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.939 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.940 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.940 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.940 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.940 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.940 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.941 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.941 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-08-11T15:56:47.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-11T15:56:47.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.942 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.942 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:47.943 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.943 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.943 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.944 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.945 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-11T15:56:47.945 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-11T15:56:47.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.945 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.946 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.946 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.946 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.946 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.947 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.947 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.947 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.948 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-11T15:56:47.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-11T15:56:47.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.948 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.949 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.950 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.950 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.950 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.950 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.951 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-11T15:56:47.951 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-11T15:56:47.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.952 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.953 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.953 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.953 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.954 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.954 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-11T15:56:47.954 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-11T15:56:47.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.955 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.955 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.955 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.955 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.956 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.956 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.956 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.957 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-08-11T15:56:47.957 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-11T15:56:47.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-11T15:56:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-11T15:56:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-11T15:56:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-11T15:56:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-11T15:56:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-11T15:56:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-11T15:56:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-11T15:56:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-11T15:56:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-11T15:56:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-11T15:56:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-11T15:56:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-11T15:56:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-11T15:56:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-11T15:56:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-11T15:56:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-08-11T15:56:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-11T15:56:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-11T15:56:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-11T15:56:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-11T15:56:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-11T15:56:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-11T15:56:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-11T15:56:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-11T15:56:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-08-11T15:56:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-11T15:56:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-11T15:56:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-08-11T15:56:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-08-11T15:56:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-11T15:56:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-11T15:56:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-11T15:56:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-08-11T15:56:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-11T15:56:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-08-11T15:56:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-11T15:56:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-11T15:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-11T15:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-11T15:56:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-11T15:56:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-11T15:56:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-11T15:56:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-08-11T15:56:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-08-11T15:56:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-08-11T15:56:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-08-11T15:56:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-08-11T15:56:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-08-11T15:56:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-08-11T15:56:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-08-11T15:56:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-08-11T15:56:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-08-11T15:56:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-08-11T15:56:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-08-11T15:56:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-08-11T15:56:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-08-11T15:56:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-08-11T15:56:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-08-11T15:56:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-08-11T15:56:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-08-11T15:56:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-08-11T15:56:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-08-11T15:56:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-08-11T15:56:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-08-11T15:56:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-08-11T15:56:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-08-11T15:56:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.057 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.057 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.057 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.057 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-08-11T15:56:48.058 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-08-11T15:56:48.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.058 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.058 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-08-11T15:56:48.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.059 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.059 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.059 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-08-11T15:56:48.059 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.060 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.060 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.060 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.061 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-08-11T15:56:48.061 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-08-11T15:56:48.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.062 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-08-11T15:56:48.062 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-08-11T15:56:48.062 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-08-11T15:56:48.062 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.062 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.063 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-08-11T15:56:48.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-08-11T15:56:48.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.064 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.065 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.065 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.065 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.066 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-08-11T15:56:48.067 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.067 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.067 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.067 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-08-11T15:56:48.068 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-08-11T15:56:48.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.068 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.069 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.069 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.069 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.069 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-08-11T15:56:48.069 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.070 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.070 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.071 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.072 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.072 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.072 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.072 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.072 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.073 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.073 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.073 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.073 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.073 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.074 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.074 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.074 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.075 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.075 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.076 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.076 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.076 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.077 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.077 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.078 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.078 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.078 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.079 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.079 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.079 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.079 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.079 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.080 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.080 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.080 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.080 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.081 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.081 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.082 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.082 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.082 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.082 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.083 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.083 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.083 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.084 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-08-11T15:56:48.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-08-11T15:56:48.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.084 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-08-11T15:56:48.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.085 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-11T15:56:48.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.086 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.086 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-08-11T15:56:48.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-08-11T15:56:48.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-08-11T15:56:48.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.088 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.088 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.088 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-11T15:56:48.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-08-11T15:56:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-08-11T15:56:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-08-11T15:56:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-08-11T15:56:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-08-11T15:56:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-08-11T15:56:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-08-11T15:56:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-11T15:56:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-08-11T15:56:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-08-11T15:56:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-11T15:56:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-08-11T15:56:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-08-11T15:56:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-08-11T15:56:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-08-11T15:56:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-11T15:56:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-08-11T15:56:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-08-11T15:56:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-08-11T15:56:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-08-11T15:56:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-08-11T15:56:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-08-11T15:56:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-08-11T15:56:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-08-11T15:56:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-08-11T15:56:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-08-11T15:56:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-08-11T15:56:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-08-11T15:56:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-08-11T15:56:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-11T15:56:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-08-11T15:56:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-08-11T15:56:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-08-11T15:56:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-11T15:56:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-08-11T15:56:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-08-11T15:56:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-08-11T15:56:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-08-11T15:56:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-08-11T15:56:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-11T15:56:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-08-11T15:56:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-08-11T15:56:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-08-11T15:56:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-08-11T15:56:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-08-11T15:56:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-08-11T15:56:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-08-11T15:56:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-08-11T15:56:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-08-11T15:56:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-08-11T15:56:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T15:56:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-08-11T15:56:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-08-11T15:56:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-08-11T15:56:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-11T15:56:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-08-11T15:56:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-08-11T15:56:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-08-11T15:56:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-08-11T15:56:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-08-11T15:56:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-08-11T15:56:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T15:56:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-08-11T15:56:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-08-11T15:56:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2023-08-11T15:56:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2023-08-11T15:56:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2023-08-11T15:56:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2023-08-11T15:56:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-11T15:56:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2023-08-11T15:56:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2023-08-11T15:56:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2023-08-11T15:56:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-11T15:56:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-08-11T15:56:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-08-11T15:56:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-08-11T15:56:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-08-11T15:56:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-08-11T15:56:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-11T15:56:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-08-11T15:56:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-08-11T15:56:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-08-11T15:56:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-11T15:56:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-08-11T15:56:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-08-11T15:56:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-08-11T15:56:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-08-11T15:56:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-08-11T15:56:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-08-11T15:56:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-08-11T15:56:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-08-11T15:56:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-08-11T15:56:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-08-11T15:56:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-08-11T15:56:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-08-11T15:56:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-08-11T15:56:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-08-11T15:56:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-11T15:56:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-11T15:56:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-08-11T15:56:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.255 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.255 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.255 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-08-11T15:56:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-08-11T15:56:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-08-11T15:56:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-08-11T15:56:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-08-11T15:56:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-11T15:56:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-08-11T15:56:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-08-11T15:56:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-08-11T15:56:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-08-11T15:56:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-08-11T15:56:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-08-11T15:56:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-08-11T15:56:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-08-11T15:56:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-08-11T15:56:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "noautoscale": { 2023-08-11T15:56:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "noautoscale", 2023-08-11T15:56:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "global autoscale flag", 2023-08-11T15:56:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-11T15:56:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-08-11T15:56:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-08-11T15:56:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-08-11T15:56:48.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-08-11T15:56:48.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.366 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-08-11T15:56:48.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-11T15:56:48.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.368 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-08-11T15:56:48.368 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.368 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.369 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.369 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-08-11T15:56:48.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-08-11T15:56:48.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.369 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.370 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-11T15:56:48.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.371 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.372 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-08-11T15:56:48.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-08-11T15:56:48.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.373 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.373 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.374 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.374 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.374 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.376 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.376 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.376 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.376 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.376 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.377 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.377 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.377 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.377 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.377 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.379 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.379 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.380 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.380 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.380 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.380 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.380 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.381 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.381 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.382 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.382 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.382 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.383 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.383 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.383 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.383 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.384 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.384 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.384 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.384 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.384 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.385 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.385 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.385 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.385 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.386 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.386 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.386 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.387 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.387 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.387 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.387 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.388 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.388 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-08-11T15:56:48.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-08-11T15:56:48.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.389 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-08-11T15:56:48.389 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.390 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-08-11T15:56:48.390 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.390 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.391 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:48.391 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:48.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.391 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-08-11T15:56:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-08-11T15:56:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-08-11T15:56:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-08-11T15:56:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-08-11T15:56:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-08-11T15:56:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-11T15:56:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-11T15:56:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.405 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.405 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.405 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.406 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.406 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.406 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.406 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.406 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.407 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.407 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.407 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.408 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.408 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.409 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.409 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.409 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.409 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.409 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.410 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.410 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.410 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.410 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.410 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.411 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.411 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.411 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.412 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.412 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.413 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.413 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.413 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.413 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.414 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.414 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-08-11T15:56:48.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-08-11T15:56:48.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.414 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.415 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.416 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.416 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.417 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-11T15:56:48.417 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-11T15:56:48.417 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.417 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.417 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.418 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-08-11T15:56:48.418 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.418 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.418 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.418 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-08-11T15:56:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-08-11T15:56:48.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.420 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.420 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-08-11T15:56:48.420 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.421 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.421 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.421 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.421 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.422 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.422 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:48.422 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:48.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.423 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.423 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.423 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-08-11T15:56:48.423 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.423 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.424 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.424 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-11T15:56:48.424 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.424 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.425 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:48.425 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:48.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.425 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.426 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-08-11T15:56:48.426 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.426 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.426 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.426 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-11T15:56:48.427 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.427 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.427 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.427 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-08-11T15:56:48.428 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-08-11T15:56:48.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.428 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.428 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-08-11T15:56:48.428 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.429 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.429 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.430 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.430 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-08-11T15:56:48.430 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-08-11T15:56:48.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.431 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.431 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.431 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-08-11T15:56:48.431 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.431 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.432 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.432 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-08-11T15:56:48.432 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-08-11T15:56:48.432 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.432 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.433 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.433 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.433 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-08-11T15:56:48.434 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-08-11T15:56:48.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.434 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.434 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-08-11T15:56:48.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-08-11T15:56:48.435 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-08-11T15:56:48.435 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.435 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.435 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.436 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.437 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-08-11T15:56:48.437 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.437 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.437 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.437 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.438 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.438 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.438 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.439 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.439 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.439 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.439 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.439 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.440 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.440 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.440 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.440 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.440 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.441 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.441 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.441 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.442 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.443 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.443 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.443 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.443 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.444 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.444 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.444 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.445 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.446 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.446 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.446 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.446 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.447 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.447 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.447 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.448 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.448 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.448 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.448 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.448 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.449 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.449 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.449 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.450 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.450 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.450 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.451 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.451 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.451 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.451 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.452 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-08-11T15:56:48.453 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-08-11T15:56:48.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.453 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.454 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:48.454 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.454 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.454 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.454 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.455 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.455 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.455 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-08-11T15:56:48.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-08-11T15:56:48.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.456 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.457 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.457 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.457 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.457 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.458 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.458 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-08-11T15:56:48.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-08-11T15:56:48.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.459 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.459 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.460 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.460 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.460 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.460 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.460 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-08-11T15:56:48.462 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.462 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.462 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.462 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-08-11T15:56:48.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-08-11T15:56:48.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.463 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.463 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.463 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.464 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.464 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.465 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.465 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-08-11T15:56:48.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-08-11T15:56:48.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.466 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.466 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.466 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.467 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.467 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.467 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.467 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.468 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.468 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.469 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.470 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.470 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.470 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.470 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.470 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.470 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.471 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.471 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.471 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.471 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.471 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.472 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.472 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.472 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.473 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.473 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.473 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.474 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.474 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.474 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.474 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.474 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.475 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.475 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.475 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.476 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.476 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.476 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.476 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.477 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.477 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.477 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.477 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.478 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.478 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.478 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.478 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.479 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.479 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.479 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.479 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.480 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.480 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.480 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.480 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.481 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.481 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.481 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.482 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:48.482 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:48.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.482 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.482 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.483 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.483 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.483 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.483 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.483 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.484 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.484 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.484 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:48.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:48.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.486 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.486 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.486 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.486 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.486 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.487 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.487 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.487 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.488 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-08-11T15:56:48.488 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.488 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.489 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.489 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.489 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.489 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.490 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.490 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.490 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.490 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.491 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.491 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.491 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.491 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.491 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.491 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.492 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.492 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.492 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.492 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.493 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.493 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.494 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.494 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.494 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.495 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.495 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.495 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.496 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.496 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.496 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.497 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.497 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.497 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.497 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.498 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.498 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.498 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.498 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.498 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.499 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.499 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.499 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.499 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.500 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.502 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.503 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.504 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-08-11T15:56:48.504 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.504 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.504 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.504 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-08-11T15:56:48.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-08-11T15:56:48.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-08-11T15:56:48.506 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.506 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.506 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.506 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-11T15:56:48.506 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.507 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.507 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.507 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.508 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.509 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.509 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.509 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.510 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.510 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.510 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.510 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.510 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.510 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.511 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.511 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.511 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.512 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.512 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.513 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.513 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.513 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.513 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.514 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.514 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.515 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.515 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.516 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.516 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.516 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.516 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.516 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.516 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.517 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.517 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.518 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.518 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.519 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.519 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.519 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.520 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.520 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.520 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.520 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.521 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-08-11T15:56:48.521 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-08-11T15:56:48.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.521 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.522 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-08-11T15:56:48.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.522 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.522 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.523 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-11T15:56:48.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.523 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-08-11T15:56:48.525 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.525 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.526 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.526 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.526 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.527 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.527 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.527 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.527 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.528 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.528 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.528 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.529 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.529 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.529 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.529 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.529 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.530 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.530 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.530 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.530 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.531 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.531 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.532 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.532 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.533 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.533 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.534 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.534 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.535 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.535 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.536 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.536 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.537 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.537 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.538 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.538 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.538 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.539 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.539 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.539 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.540 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.540 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.540 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.541 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.541 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.542 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.542 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.543 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.544 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.545 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.545 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.545 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.546 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.547 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-08-11T15:56:48.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-08-11T15:56:48.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.549 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.550 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.550 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.550 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.551 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.552 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-08-11T15:56:48.552 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-08-11T15:56:48.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.553 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.553 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-08-11T15:56:48.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.554 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.554 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.554 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.555 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.555 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.556 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-08-11T15:56:48.556 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-08-11T15:56:48.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.558 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.558 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.559 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.559 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.559 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.561 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-08-11T15:56:48.561 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-08-11T15:56:48.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.562 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.563 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.565 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-08-11T15:56:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-08-11T15:56:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.566 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.567 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-08-11T15:56:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-08-11T15:56:48.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.571 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-08-11T15:56:48.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-08-11T15:56:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-08-11T15:56:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-08-11T15:56:48.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.576 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.577 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-08-11T15:56:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-08-11T15:56:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:48.579 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-08-11T15:56:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.580 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-08-11T15:56:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-08-11T15:56:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.581 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.582 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.583 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-08-11T15:56:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-08-11T15:56:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-08-11T15:56:48.586 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-08-11T15:56:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.587 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-08-11T15:56:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.590 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-08-11T15:56:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-08-11T15:56:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-08-11T15:56:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-08-11T15:56:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-08-11T15:56:48.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.594 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-08-11T15:56:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.595 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.596 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.597 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.598 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.599 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.600 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.601 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.604 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.605 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-08-11T15:56:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-08-11T15:56:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-08-11T15:56:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-08-11T15:56:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-08-11T15:56:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-11T15:56:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-08-11T15:56:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-08-11T15:56:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-08-11T15:56:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-08-11T15:56:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-11T15:56:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-08-11T15:56:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-08-11T15:56:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-11T15:56:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-08-11T15:56:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-08-11T15:56:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-11T15:56:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-08-11T15:56:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-08-11T15:56:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-11T15:56:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-08-11T15:56:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-08-11T15:56:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-11T15:56:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-08-11T15:56:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-08-11T15:56:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-08-11T15:56:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-08-11T15:56:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-08-11T15:56:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-08-11T15:56:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-11T15:56:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-08-11T15:56:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-08-11T15:56:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.707 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-08-11T15:56:48.707 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-08-11T15:56:48.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.708 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.708 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.709 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.709 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.710 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.711 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-08-11T15:56:48.711 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-08-11T15:56:48.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.712 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.713 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-08-11T15:56:48.713 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.714 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.714 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.714 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.715 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.715 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.717 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-08-11T15:56:48.717 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-08-11T15:56:48.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.718 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.718 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.718 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.719 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.719 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.720 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.720 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.720 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-08-11T15:56:48.721 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-08-11T15:56:48.721 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.721 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.721 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.722 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.722 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.722 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.722 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.722 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.723 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.723 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.723 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.724 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.724 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.725 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.725 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.725 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.725 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.725 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.726 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.726 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.726 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.726 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.727 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.727 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.727 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.727 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.728 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.728 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.729 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.729 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.729 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.730 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.730 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.730 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.730 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.731 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.731 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.731 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.732 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.732 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.732 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.732 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.733 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.733 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.733 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.733 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.733 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.734 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.734 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.734 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.734 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.734 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.735 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.735 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.735 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.736 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.736 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.736 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.736 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.737 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.737 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.737 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.738 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-08-11T15:56:48.738 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-08-11T15:56:48.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.739 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.739 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.739 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.740 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.740 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.740 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.741 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.741 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-08-11T15:56:48.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-08-11T15:56:48.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.742 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.742 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.743 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.743 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.743 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.743 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.744 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.744 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-08-11T15:56:48.744 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-08-11T15:56:48.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.745 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.745 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-11T15:56:48.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.745 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.746 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.746 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.746 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.748 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-08-11T15:56:48.748 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.748 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.749 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.749 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.749 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.751 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.751 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.751 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.751 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.751 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.752 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.752 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.752 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.752 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.753 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.753 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.753 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.754 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.754 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.755 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.755 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.755 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.755 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.756 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.756 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.756 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.756 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-08-11T15:56:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-08-11T15:56:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-08-11T15:56:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-08-11T15:56:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-11T15:56:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-08-11T15:56:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-08-11T15:56:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-11T15:56:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-08-11T15:56:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-08-11T15:56:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-08-11T15:56:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-08-11T15:56:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-08-11T15:56:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-08-11T15:56:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-08-11T15:56:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-08-11T15:56:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-08-11T15:56:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-08-11T15:56:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-08-11T15:56:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-08-11T15:56:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-08-11T15:56:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-11T15:56:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 25319, 2023-08-11T15:56:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "y", 2023-08-11T15:56:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540138322906710015, 2023-08-11T15:56:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-08-11T15:56:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-08-11T15:56:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-08-11T15:56:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-08-11T15:56:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-08-11T15:56:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-08-11T15:56:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-08-11T15:56:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-08-11T15:56:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-08-11T15:56:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-08-11T15:56:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-08-11T15:56:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-08-11T15:56:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-08-11T15:56:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-08-11T15:56:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-08-11T15:56:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-08-11T15:56:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-08-11T15:56:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-08-11T15:56:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-08-11T15:56:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-08-11T15:56:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-08-11T15:56:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-08-11T15:56:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-08-11T15:56:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-08-11T15:56:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-08-11T15:56:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-11T15:56:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-08-11T15:56:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-08-11T15:56:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-08-11T15:56:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-08-11T15:56:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-08-11T15:56:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-08-11T15:56:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-08-11T15:56:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-08-11T15:56:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-08-11T15:56:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-08-11T15:56:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-08-11T15:56:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T15:56:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-08-11T15:56:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-08-11T15:56:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-08-11T15:56:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-08-11T15:56:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-11T15:56:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T15:56:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-08-11T15:56:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-08-11T15:56:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-08-11T15:56:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-08-11T15:56:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-11T15:56:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-08-11T15:56:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-08-11T15:56:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-08-11T15:56:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-11T15:56:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-11T15:56:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-08-11T15:56:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-08-11T15:56:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-08-11T15:56:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-08-11T15:56:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-08-11T15:56:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-08-11T15:56:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-11T15:56:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-08-11T15:56:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-08-11T15:56:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-08-11T15:56:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-08-11T15:56:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T15:56:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-08-11T15:56:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-08-11T15:56:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-08-11T15:56:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-08-11T15:56:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-11T15:56:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T15:56:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-08-11T15:56:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-08-11T15:56:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-11T15:56:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-08-11T15:56:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-08-11T15:56:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-08-11T15:56:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-08-11T15:56:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-08-11T15:56:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-08-11T15:56:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-08-11T15:56:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-08-11T15:56:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-08-11T15:56:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-11T15:56:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-11T15:56:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-08-11T15:56:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-08-11T15:56:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-11T15:56:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-11T15:56:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-08-11T15:56:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-08-11T15:56:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-11T15:56:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-08-11T15:56:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:48.962 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-08-11T15:56:48.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-08-11T15:56:48.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.962 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-08-11T15:56:48.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.963 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-11T15:56:48.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.964 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.965 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-08-11T15:56:48.965 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-08-11T15:56:48.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.965 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.966 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-08-11T15:56:48.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.966 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.966 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-11T15:56:48.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.968 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.968 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-08-11T15:56:48.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-08-11T15:56:48.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:48.969 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.969 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.969 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-08-11T15:56:48.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.970 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.970 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.970 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-11T15:56:48.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.972 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-08-11T15:56:48.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-08-11T15:56:48.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.973 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.973 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.973 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-11T15:56:48.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-11T15:56:48.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.975 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-08-11T15:56:48.975 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-08-11T15:56:48.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:48.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.976 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:48.976 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.976 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.976 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.976 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-11T15:56:48.977 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.977 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.977 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-08-11T15:56:48.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-08-11T15:56:48.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:48.978 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.978 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.978 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-08-11T15:56:48.979 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.979 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.979 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.979 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-11T15:56:48.979 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.980 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.980 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-08-11T15:56:48.981 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-08-11T15:56:48.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.981 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.981 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.982 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.982 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.982 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-11T15:56:48.982 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.983 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.983 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-08-11T15:56:48.983 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-08-11T15:56:48.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.984 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:48.984 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.984 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.985 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-11T15:56:48.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.985 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-08-11T15:56:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-08-11T15:56:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-11T15:56:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-08-11T15:56:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-08-11T15:56:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-11T15:56:48.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.991 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.991 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.991 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:48.992 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.993 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-08-11T15:56:48.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-08-11T15:56:48.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.994 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:48.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-11T15:56:48.995 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:48.995 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:48.995 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:48.996 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-08-11T15:56:48.996 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:48.996 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:48.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:48.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:48.997 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-08-11T15:56:48.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-08-11T15:56:48.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:48.998 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:48.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:48.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-11T15:56:48.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.000 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.000 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-08-11T15:56:49.001 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.001 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.002 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-08-11T15:56:49.002 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-08-11T15:56:49.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.004 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-11T15:56:49.004 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.004 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.005 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:49.005 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.006 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-08-11T15:56:49.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-08-11T15:56:49.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.007 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-11T15:56:49.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.008 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.008 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-08-11T15:56:49.008 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.009 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.009 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-08-11T15:56:49.009 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-08-11T15:56:49.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.010 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.010 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-11T15:56:49.010 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.011 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.011 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.011 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-08-11T15:56:49.012 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.012 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.013 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-08-11T15:56:49.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-08-11T15:56:49.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.013 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.014 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-11T15:56:49.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.015 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-08-11T15:56:49.015 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.015 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.016 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-08-11T15:56:49.016 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-08-11T15:56:49.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.016 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.017 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-11T15:56:49.017 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.017 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.017 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.018 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-08-11T15:56:49.018 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.018 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.019 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-08-11T15:56:49.019 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-08-11T15:56:49.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.019 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-11T15:56:49.020 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.020 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.020 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.020 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-08-11T15:56:49.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.021 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.021 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-08-11T15:56:49.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-08-11T15:56:49.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.022 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.023 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-11T15:56:49.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.023 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.023 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.023 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-08-11T15:56:49.024 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.024 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.024 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-08-11T15:56:49.025 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-08-11T15:56:49.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.025 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.025 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-11T15:56:49.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.026 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.026 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.026 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:49.027 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.027 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.027 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-08-11T15:56:49.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-08-11T15:56:49.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.028 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.1", 2023-08-11T15:56:49.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.029 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.029 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-08-11T15:56:49.030 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.030 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.030 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-08-11T15:56:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-08-11T15:56:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-11T15:56:49.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.032 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.032 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:49.032 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.033 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.033 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-08-11T15:56:49.034 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-08-11T15:56:49.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.034 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-11T15:56:49.035 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.035 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.035 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.035 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-08-11T15:56:49.035 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.036 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.036 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-08-11T15:56:49.036 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-08-11T15:56:49.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.037 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.037 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-11T15:56:49.037 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.038 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.038 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.038 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-08-11T15:56:49.038 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.039 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.040 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-08-11T15:56:49.040 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-08-11T15:56:49.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.041 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.042 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-08-11T15:56:49.044 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.045 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.045 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-08-11T15:56:49.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-08-11T15:56:49.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:49.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.046 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-11T15:56:49.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.048 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-08-11T15:56:49.048 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-08-11T15:56:49.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.049 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.049 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-08-11T15:56:49.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.050 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.050 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-11T15:56:49.050 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.051 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.051 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-08-11T15:56:49.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-08-11T15:56:49.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.052 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-08-11T15:56:49.053 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.053 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.053 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.053 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-11T15:56:49.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.054 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.054 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-08-11T15:56:49.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-08-11T15:56:49.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.055 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.055 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-08-11T15:56:49.055 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.055 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.056 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-08-11T15:56:49.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.057 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.057 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-08-11T15:56:49.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-08-11T15:56:49.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.058 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.058 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.058 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.058 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.059 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-11T15:56:49.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.059 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.060 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-08-11T15:56:49.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-08-11T15:56:49.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:49.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.062 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.062 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.062 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-08-11T15:56:49.062 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.063 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-08-11T15:56:49.063 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-08-11T15:56:49.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.064 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:49.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.065 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.065 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-08-11T15:56:49.066 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.066 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.066 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.067 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.068 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.069 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.069 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.069 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.069 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.070 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.070 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.070 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.070 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.071 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.072 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.073 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-08-11T15:56:49.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-08-11T15:56:49.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.074 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.075 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.077 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-11T15:56:49.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.078 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.079 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.079 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.080 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.081 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.081 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-11T15:56:49.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.083 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.084 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.084 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.084 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.086 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.086 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.087 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.087 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.087 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.088 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.088 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.089 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.089 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.090 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.090 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.090 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.092 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-11T15:56:49.092 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-11T15:56:49.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.093 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.093 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.094 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-11T15:56:49.094 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.094 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.095 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-11T15:56:49.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-11T15:56:49.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.095 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-08-11T15:56:49.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.097 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-11T15:56:49.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-08-11T15:56:49.098 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-08-11T15:56:49.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.099 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.099 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:49.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.100 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.101 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-08-11T15:56:49.101 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.101 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.102 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-08-11T15:56:49.102 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-08-11T15:56:49.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.102 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:49.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.103 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-11T15:56:49.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.105 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-08-11T15:56:49.105 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-08-11T15:56:49.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.106 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.106 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.106 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.106 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.106 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-08-11T15:56:49.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.107 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-08-11T15:56:49.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-08-11T15:56:49.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.108 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-08-11T15:56:49.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.109 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.110 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-08-11T15:56:49.110 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-08-11T15:56:49.110 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.110 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-08-11T15:56:49.110 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.112 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-08-11T15:56:49.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-08-11T15:56:49.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.114 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-11T15:56:49.115 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.115 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.115 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.116 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-11T15:56:49.116 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.118 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-08-11T15:56:49.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-08-11T15:56:49.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.119 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.120 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.120 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-11T15:56:49.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.121 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-08-11T15:56:49.121 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-08-11T15:56:49.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.122 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.122 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.122 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.123 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-11T15:56:49.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.124 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.124 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-08-11T15:56:49.124 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-08-11T15:56:49.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.125 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.125 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.125 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.125 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.126 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.126 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-11T15:56:49.126 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.126 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.126 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.127 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-08-11T15:56:49.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-08-11T15:56:49.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.127 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.128 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.129 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-11T15:56:49.129 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.130 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-08-11T15:56:49.130 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-08-11T15:56:49.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.130 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.132 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-11T15:56:49.132 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.132 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.133 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-08-11T15:56:49.133 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-08-11T15:56:49.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.135 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-08-11T15:56:49.135 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.135 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.136 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.136 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-08-11T15:56:49.136 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-08-11T15:56:49.139 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-08-11T15:56:49.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.140 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.140 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.140 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.141 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.141 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.141 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.141 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-11T15:56:49.141 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.142 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-08-11T15:56:49.142 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-08-11T15:56:49.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.144 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.144 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.144 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-11T15:56:49.144 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.144 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.145 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-08-11T15:56:49.145 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-08-11T15:56:49.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.146 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-11T15:56:49.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.148 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-08-11T15:56:49.148 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-08-11T15:56:49.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.149 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.149 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.149 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.149 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.150 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-08-11T15:56:49.150 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.150 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.151 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-08-11T15:56:49.151 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-08-11T15:56:49.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.151 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.152 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.152 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.152 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.153 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-11T15:56:49.153 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.153 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.153 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.154 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-08-11T15:56:49.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-08-11T15:56:49.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.154 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.155 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.155 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.155 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-11T15:56:49.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-08-11T15:56:49.157 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.158 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.158 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.158 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.158 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.159 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.159 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.160 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.160 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.160 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.160 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.160 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.161 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.161 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.162 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.163 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.163 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.164 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.164 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.164 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.164 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.165 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.165 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.166 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.166 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.167 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.167 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.167 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.167 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.168 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.168 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.168 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.168 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.168 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.169 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.170 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.171 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.171 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.171 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.171 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.172 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.172 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-08-11T15:56:49.172 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-08-11T15:56:49.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.173 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.173 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-08-11T15:56:49.173 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.173 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.173 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-11T15:56:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-08-11T15:56:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-08-11T15:56:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-08-11T15:56:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-11T15:56:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-08-11T15:56:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-11T15:56:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-11T15:56:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-08-11T15:56:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-11T15:56:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-11T15:56:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-11T15:56:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-08-11T15:56:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-08-11T15:56:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-11T15:56:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-11T15:56:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-08-11T15:56:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-11T15:56:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-11T15:56:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-11T15:56:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-11T15:56:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-11T15:56:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-11T15:56:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-11T15:56:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-11T15:56:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-11T15:56:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-11T15:56:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-11T15:56:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-11T15:56:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-11T15:56:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-11T15:56:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-11T15:56:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-11T15:56:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-11T15:56:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-08-11T15:56:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-11T15:56:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-11T15:56:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-11T15:56:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-08-11T15:56:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-08-11T15:56:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-08-11T15:56:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-11T15:56:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-11T15:56:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-11T15:56:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-11T15:56:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-11T15:56:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-11T15:56:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-11T15:56:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-11T15:56:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-11T15:56:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-08-11T15:56:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-11T15:56:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-11T15:56:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-11T15:56:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-11T15:56:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-11T15:56:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-11T15:56:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-11T15:56:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-11T15:56:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-11T15:56:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-11T15:56:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-11T15:56:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-11T15:56:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-11T15:56:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-11T15:56:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-11T15:56:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-11T15:56:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-11T15:56:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-08-11T15:56:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-11T15:56:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-11T15:56:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-11T15:56:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-11T15:56:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-11T15:56:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-11T15:56:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-11T15:56:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-11T15:56:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-08-11T15:56:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-11T15:56:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-11T15:56:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-08-11T15:56:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-08-11T15:56:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-11T15:56:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-11T15:56:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-11T15:56:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-08-11T15:56:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-11T15:56:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-08-11T15:56:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-11T15:56:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-11T15:56:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-11T15:56:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-11T15:56:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-11T15:56:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-11T15:56:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-11T15:56:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-08-11T15:56:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-08-11T15:56:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-08-11T15:56:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-08-11T15:56:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-08-11T15:56:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-08-11T15:56:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-08-11T15:56:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-08-11T15:56:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-08-11T15:56:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-08-11T15:56:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-08-11T15:56:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-08-11T15:56:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-08-11T15:56:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-08-11T15:56:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-08-11T15:56:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-08-11T15:56:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-08-11T15:56:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-08-11T15:56:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-08-11T15:56:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-08-11T15:56:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-08-11T15:56:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-08-11T15:56:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-08-11T15:56:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-08-11T15:56:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-08-11T15:56:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-08-11T15:56:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-08-11T15:56:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-08-11T15:56:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-08-11T15:56:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-08-11T15:56:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-08-11T15:56:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-08-11T15:56:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-08-11T15:56:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-08-11T15:56:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-08-11T15:56:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-08-11T15:56:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-08-11T15:56:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-08-11T15:56:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-08-11T15:56:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-08-11T15:56:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-08-11T15:56:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-08-11T15:56:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-11T15:56:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-08-11T15:56:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-08-11T15:56:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-08-11T15:56:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-11T15:56:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-08-11T15:56:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-08-11T15:56:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-08-11T15:56:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-08-11T15:56:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-08-11T15:56:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-08-11T15:56:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-08-11T15:56:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-11T15:56:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-08-11T15:56:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-08-11T15:56:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-11T15:56:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-08-11T15:56:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.410 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-08-11T15:56:49.410 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-08-11T15:56:49.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.410 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.411 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-08-11T15:56:49.411 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.411 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.411 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.412 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-11T15:56:49.412 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.412 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.412 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.413 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-08-11T15:56:49.413 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.414 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.414 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.414 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.415 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.416 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.416 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.416 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.416 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.417 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.417 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.417 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.417 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.417 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.418 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.418 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.418 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.418 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.419 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.419 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.420 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.420 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.420 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.421 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.421 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.421 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.422 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.422 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.422 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.423 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.423 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.423 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.423 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.424 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.424 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.424 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.424 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.425 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.425 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.425 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.425 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.425 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.426 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.426 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.426 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.427 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.427 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.427 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.428 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.428 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.428 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.428 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.429 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.429 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-08-11T15:56:49.429 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-08-11T15:56:49.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.430 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.430 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-08-11T15:56:49.430 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.430 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.431 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.431 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.431 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.431 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.432 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-08-11T15:56:49.432 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-08-11T15:56:49.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.432 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.433 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.433 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-08-11T15:56:49.433 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.433 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.434 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.435 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.435 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:49.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:49.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.436 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:49.436 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.437 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.437 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.437 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.437 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.438 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-08-11T15:56:49.439 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-08-11T15:56:49.439 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-08-11T15:56:49.439 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.439 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-08-11T15:56:49.439 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-08-11T15:56:49.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.440 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.440 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-08-11T15:56:49.440 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-11T15:56:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.442 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.442 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-08-11T15:56:49.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-08-11T15:56:49.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.443 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.443 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-08-11T15:56:49.443 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.443 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.444 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.444 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-11T15:56:49.444 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.445 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.445 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-08-11T15:56:49.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-08-11T15:56:49.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.446 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.446 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.446 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.446 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.447 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.447 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-08-11T15:56:49.447 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.447 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.447 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.448 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:49.448 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:49.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.448 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-08-11T15:56:49.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-08-11T15:56:49.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.450 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-11T15:56:49.450 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.450 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.451 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.452 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.452 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.452 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.452 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.452 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.453 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.453 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.453 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.453 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.453 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.454 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.454 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.454 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.454 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.455 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.455 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.455 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.456 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.456 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.456 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.457 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.457 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.458 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.458 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.459 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.459 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.459 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.460 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.460 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.460 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.460 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.460 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.461 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.461 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.462 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.462 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.463 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.463 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.463 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.463 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.463 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.464 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.465 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-08-11T15:56:49.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-08-11T15:56:49.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.465 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.466 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.466 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-08-11T15:56:49.467 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.467 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.467 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-08-11T15:56:49.468 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-08-11T15:56:49.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.469 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-08-11T15:56:49.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.469 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.469 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-08-11T15:56:49.470 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.470 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.470 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.470 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-08-11T15:56:49.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-08-11T15:56:49.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-08-11T15:56:49.472 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.472 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.472 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T15:56:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-08-11T15:56:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-08-11T15:56:49.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.474 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.474 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:49.474 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:49.475 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-08-11T15:56:49.475 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.475 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-11T15:56:49.475 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.476 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.476 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-08-11T15:56:49.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-08-11T15:56:49.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.477 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.477 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.477 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.477 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.478 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.478 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-08-11T15:56:49.478 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.479 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-08-11T15:56:49.479 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-08-11T15:56:49.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.479 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.480 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-08-11T15:56:49.480 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.481 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.481 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.481 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T15:56:49.482 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.483 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-08-11T15:56:49.485 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.486 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.486 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.487 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.487 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.488 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.489 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.489 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.489 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.490 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.490 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.491 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.491 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.491 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.491 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.492 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.492 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.492 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.492 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.493 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.493 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.493 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.493 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.494 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.494 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.494 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.495 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.495 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.495 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.495 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.496 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.496 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.496 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.497 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.497 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.497 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.497 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.498 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.498 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.498 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.498 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.498 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.499 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.499 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.499 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.499 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.500 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.502 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-08-11T15:56:49.504 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.504 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.504 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.504 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.504 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.505 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.506 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.506 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.506 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.506 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.506 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.507 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.507 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.507 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.507 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.508 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.508 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.509 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.509 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.509 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.510 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.511 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.511 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.511 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.512 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.512 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.512 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.513 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.513 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.513 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.513 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.514 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.514 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.514 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.514 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.515 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.515 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.515 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.515 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.515 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.516 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.516 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.516 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.517 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.518 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.519 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.519 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.519 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.520 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.520 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.521 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2023-08-11T15:56:49.521 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.521 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.522 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.522 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2023-08-11T15:56:49.522 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2023-08-11T15:56:49.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.522 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.523 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.523 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2023-08-11T15:56:49.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.523 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.523 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.524 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-11T15:56:49.524 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.525 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.525 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2023-08-11T15:56:49.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2023-08-11T15:56:49.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.526 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.526 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:49.527 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2023-08-11T15:56:49.527 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.527 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.528 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-11T15:56:49.528 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.529 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.529 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.530 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.530 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.530 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.531 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.531 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.531 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.531 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.532 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.532 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.532 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.532 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.532 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.533 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.533 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.533 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.534 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.534 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.534 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.535 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.535 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.535 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.535 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.536 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.536 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.536 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.537 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.537 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.537 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.538 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.538 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.538 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.538 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.538 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.539 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.539 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.539 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.539 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.539 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.540 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.540 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.540 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.541 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.541 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.541 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.541 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.542 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.542 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.542 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.542 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.543 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.544 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-08-11T15:56:49.544 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.544 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.544 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.545 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-08-11T15:56:49.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-08-11T15:56:49.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.545 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.546 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-08-11T15:56:49.546 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.546 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.546 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.546 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-08-11T15:56:49.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-11T15:56:49.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.547 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.547 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.548 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.548 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.549 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.549 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.549 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.549 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.550 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.550 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.550 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.550 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.550 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.551 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.551 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.551 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.552 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.552 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.552 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.553 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.553 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.553 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.553 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.554 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.554 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.554 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.555 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.555 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.555 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.555 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.556 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.556 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.556 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.556 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.557 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.557 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.557 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.557 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.557 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.558 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.558 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.558 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.559 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.559 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.560 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.560 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.560 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.560 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-08-11T15:56:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-08-11T15:56:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.562 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.562 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.562 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.562 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.563 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-08-11T15:56:49.563 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-11T15:56:49.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.563 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.564 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-08-11T15:56:49.564 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-08-11T15:56:49.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.564 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.565 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-08-11T15:56:49.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.565 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.565 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.565 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-08-11T15:56:49.566 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.566 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.566 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.566 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-08-11T15:56:49.567 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-08-11T15:56:49.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.567 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.567 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-08-11T15:56:49.567 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.568 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.568 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.568 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-08-11T15:56:49.568 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.569 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-08-11T15:56:49.569 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-08-11T15:56:49.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.570 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.571 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.571 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-08-11T15:56:49.571 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.571 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.572 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-08-11T15:56:49.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-08-11T15:56:49.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.572 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-08-11T15:56:49.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.573 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-11T15:56:49.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-11T15:56:49.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.574 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-08-11T15:56:49.575 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.576 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.576 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.576 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.576 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.576 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.577 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.577 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.578 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.578 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.578 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.578 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.578 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.579 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.579 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.579 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.580 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.580 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.580 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.582 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.582 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.582 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.582 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.583 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.583 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.583 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.584 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.584 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.584 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.584 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.584 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.585 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.585 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.585 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.585 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.585 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.586 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.586 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.586 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.586 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.587 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.587 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.588 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.588 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-08-11T15:56:49.590 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.590 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.591 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.591 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.591 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.591 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.592 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.592 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.592 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.593 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.593 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.593 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.593 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.593 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.594 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.594 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.594 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.594 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.595 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.595 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.595 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.596 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.596 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.596 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.596 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.597 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.597 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.597 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.597 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.598 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.598 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.598 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.599 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.599 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.599 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.599 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.599 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.599 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.600 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.600 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.600 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.600 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.600 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.601 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.601 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.601 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.601 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.602 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.602 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.602 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.603 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.603 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.603 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.604 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.604 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.604 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.604 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.605 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-08-11T15:56:49.605 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.606 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.606 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.606 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.606 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.607 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.608 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.608 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.608 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.608 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.609 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.609 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.609 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.609 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.609 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.610 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.610 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.610 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.610 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.611 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.611 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.611 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.612 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.612 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.612 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.612 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.612 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.613 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.613 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.613 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.614 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.614 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.614 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.614 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.615 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.615 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.615 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.615 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.615 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.616 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.616 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.616 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.616 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.616 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.617 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.617 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.617 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.618 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.618 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.618 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.619 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.619 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.619 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.619 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.619 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.620 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.620 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.620 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-08-11T15:56:49.621 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.621 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.621 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.621 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-08-11T15:56:49.621 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-08-11T15:56:49.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.622 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.622 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.622 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.623 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-11T15:56:49.623 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.624 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.624 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.624 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.624 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.625 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.625 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.625 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.625 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.625 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.626 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.626 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.626 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.626 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.626 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.627 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.627 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.627 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.627 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.628 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.628 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.629 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.629 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.629 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.630 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.630 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.630 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.630 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.631 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.631 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.631 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.632 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.632 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.632 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.632 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.633 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.633 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.633 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.633 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.633 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.634 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.634 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.634 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.634 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.635 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.635 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.636 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.636 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.636 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.638 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-08-11T15:56:49.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-08-11T15:56:49.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.639 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.639 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.639 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.640 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-08-11T15:56:49.640 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-08-11T15:56:49.640 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-08-11T15:56:49.640 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.640 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-08-11T15:56:49.641 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.641 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.641 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.642 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-08-11T15:56:49.643 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.643 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.643 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.643 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.643 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.644 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.644 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.645 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.645 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.645 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.645 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.645 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.646 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.646 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.646 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.646 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.646 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.647 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.647 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.647 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.648 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.648 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.648 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.649 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.649 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.649 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.649 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.651 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.651 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.651 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.651 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.652 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.652 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.652 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.652 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.652 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.653 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.653 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.653 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.653 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.653 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.654 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.654 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.654 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.655 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.655 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.655 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.656 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.656 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.656 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.656 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.656 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.657 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.658 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-08-11T15:56:49.658 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.658 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.658 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.658 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.658 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.659 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.659 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.659 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.660 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.660 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.660 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.660 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.660 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.661 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.661 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.661 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.661 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.661 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.662 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.662 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.662 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.662 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.663 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.663 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.663 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.663 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.664 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.664 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.664 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.664 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.665 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.665 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.665 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.666 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.666 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.666 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.666 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.666 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.667 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.667 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.667 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.667 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.667 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.668 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.668 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.668 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.668 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.668 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.669 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.669 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.669 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.670 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.670 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.670 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.670 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.671 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.671 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.671 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.671 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-08-11T15:56:49.673 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.673 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.673 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.673 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.673 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.674 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.674 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.674 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.675 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.675 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.675 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.675 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.675 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.675 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.681 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.681 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.681 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.681 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.681 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.681 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.682 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.682 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.682 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.682 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.682 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.683 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.683 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.683 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.684 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.684 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.684 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.685 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.685 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.685 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.685 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.685 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.686 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.686 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.686 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.686 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.686 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.687 INFO:teuthology.orchestra.run.smithi032.stdout: "noautoscale": { 2023-08-11T15:56:49.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "noautoscale", 2023-08-11T15:56:49.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.687 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.688 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.688 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.688 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.688 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.689 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "global autoscale flag", 2023-08-11T15:56:49.689 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-11T15:56:49.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.689 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.690 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:49.690 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:49.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.690 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.690 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:49.691 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.691 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.691 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.691 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.691 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.692 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.692 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-08-11T15:56:49.692 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-08-11T15:56:49.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:49.693 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.693 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-08-11T15:56:49.693 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-08-11T15:56:49.694 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.694 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.694 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-08-11T15:56:49.694 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-11T15:56:49.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.695 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.696 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-08-11T15:56:49.696 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.696 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.696 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.696 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-08-11T15:56:49.697 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-08-11T15:56:49.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.697 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.697 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.698 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.698 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.698 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.698 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-11T15:56:49.698 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.699 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.699 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-08-11T15:56:49.699 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-08-11T15:56:49.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.700 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.700 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.701 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.701 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.701 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.702 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.702 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.702 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.703 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.703 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.703 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.703 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.703 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.704 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.704 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.704 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.704 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.704 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.705 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.705 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.705 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.705 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.705 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-08-11T15:56:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-08-11T15:56:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-08-11T15:56:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-08-11T15:56:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-08-11T15:56:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-08-11T15:56:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-08-11T15:56:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-08-11T15:56:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-08-11T15:56:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-08-11T15:56:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-11T15:56:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-11T15:56:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-08-11T15:56:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-08-11T15:56:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-11T15:56:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-11T15:56:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-08-11T15:56:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-08-11T15:56:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-08-11T15:56:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-08-11T15:56:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-08-11T15:56:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-11T15:56:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-08-11T15:56:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-11T15:56:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-08-11T15:56:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-08-11T15:56:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-08-11T15:56:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-08-11T15:56:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-08-11T15:56:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-08-11T15:56:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-08-11T15:56:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-08-11T15:56:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-08-11T15:56:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-08-11T15:56:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-08-11T15:56:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-08-11T15:56:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-08-11T15:56:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-08-11T15:56:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-08-11T15:56:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-08-11T15:56:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-08-11T15:56:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-08-11T15:56:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-08-11T15:56:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-08-11T15:56:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-08-11T15:56:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-08-11T15:56:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-08-11T15:56:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-08-11T15:56:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-08-11T15:56:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-08-11T15:56:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-08-11T15:56:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-08-11T15:56:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-08-11T15:56:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-08-11T15:56:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-11T15:56:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-08-11T15:56:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-08-11T15:56:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-08-11T15:56:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-11T15:56:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-08-11T15:56:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-08-11T15:56:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-08-11T15:56:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-08-11T15:56:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-08-11T15:56:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-08-11T15:56:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-08-11T15:56:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-08-11T15:56:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-08-11T15:56:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-08-11T15:56:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-08-11T15:56:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-08-11T15:56:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-08-11T15:56:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-08-11T15:56:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-08-11T15:56:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-08-11T15:56:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-08-11T15:56:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-08-11T15:56:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-08-11T15:56:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-08-11T15:56:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-08-11T15:56:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-08-11T15:56:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-08-11T15:56:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-08-11T15:56:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-08-11T15:56:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-08-11T15:56:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-08-11T15:56:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-08-11T15:56:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-08-11T15:56:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-08-11T15:56:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-08-11T15:56:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-08-11T15:56:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-08-11T15:56:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-08-11T15:56:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-08-11T15:56:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-08-11T15:56:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-08-11T15:56:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-08-11T15:56:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-08-11T15:56:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-11T15:56:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-08-11T15:56:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-08-11T15:56:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-08-11T15:56:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-08-11T15:56:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-11T15:56:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-08-11T15:56:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-08-11T15:56:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-11T15:56:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-08-11T15:56:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-08-11T15:56:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-11T15:56:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-08-11T15:56:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-08-11T15:56:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-11T15:56:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-08-11T15:56:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-08-11T15:56:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-11T15:56:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-08-11T15:56:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-08-11T15:56:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-08-11T15:56:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-08-11T15:56:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-08-11T15:56:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-08-11T15:56:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-11T15:56:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-08-11T15:56:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-08-11T15:56:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-08-11T15:56:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-08-11T15:56:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-08-11T15:56:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-08-11T15:56:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-08-11T15:56:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-08-11T15:56:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-08-11T15:56:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-08-11T15:56:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-08-11T15:56:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-08-11T15:56:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-08-11T15:56:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-08-11T15:56:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-08-11T15:56:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-08-11T15:56:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-08-11T15:56:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-11T15:56:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-08-11T15:56:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-08-11T15:56:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-08-11T15:56:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-08-11T15:56:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-08-11T15:56:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-11T15:56:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-08-11T15:56:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-08-11T15:56:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-11T15:56:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-08-11T15:56:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-08-11T15:56:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-08-11T15:56:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-08-11T15:56:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-08-11T15:56:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-08-11T15:56:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.072 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.072 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.072 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.073 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.073 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:50.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:50.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.074 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.074 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:50.074 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.075 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.076 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.076 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.076 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.077 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.077 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.077 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.077 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.077 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.078 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.078 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.078 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.078 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.078 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.079 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.079 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.079 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.079 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.079 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.080 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.080 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.081 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.081 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.081 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.082 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.085 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.085 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.085 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.085 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.086 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.086 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.087 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.087 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.088 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.088 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.088 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.088 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.089 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.089 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.089 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.089 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.090 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-08-11T15:56:50.090 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-08-11T15:56:50.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.091 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.091 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.091 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.091 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.091 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.091 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.092 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.092 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.092 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.092 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-08-11T15:56:50.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-08-11T15:56:50.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-08-11T15:56:50.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.094 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.094 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.094 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.095 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-08-11T15:56:50.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-08-11T15:56:50.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-11T15:56:50.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.097 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.097 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.097 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.098 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.098 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:50.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.099 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.099 INFO:teuthology.orchestra.run.smithi032.stdout: "modules": [ 2023-08-11T15:56:50.099 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-08-11T15:56:50.099 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard", 2023-08-11T15:56:50.099 INFO:teuthology.orchestra.run.smithi032.stdout: "iostat", 2023-08-11T15:56:50.100 INFO:teuthology.orchestra.run.smithi032.stdout: "nfs", 2023-08-11T15:56:50.100 INFO:teuthology.orchestra.run.smithi032.stdout: "restful", 2023-08-11T15:56:50.100 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_schedule" 2023-08-11T15:56:50.100 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.100 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-08-11T15:56:50.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-08-11T15:56:50.101 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.101 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.101 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.102 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:50.102 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:50.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.102 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:50.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.103 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.103 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-08-11T15:56:50.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.105 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.106 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.106 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.106 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.106 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.106 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.107 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.107 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.107 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.107 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.107 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.108 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.108 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.109 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.109 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.110 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.110 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.110 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.110 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.111 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.113 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.113 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.114 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.114 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.114 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.114 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.115 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.116 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.116 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.117 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.117 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.117 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.117 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.118 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.118 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-08-11T15:56:50.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-08-11T15:56:50.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.119 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.120 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.120 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-08-11T15:56:50.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.121 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-08-11T15:56:50.121 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-08-11T15:56:50.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.121 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.122 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-08-11T15:56:50.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.122 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.122 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.122 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-08-11T15:56:50.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.123 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-08-11T15:56:50.124 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-08-11T15:56:50.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.124 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.125 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.125 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.125 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-08-11T15:56:50.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.126 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.126 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.126 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-08-11T15:56:50.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-08-11T15:56:50.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.127 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.127 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.127 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.127 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-08-11T15:56:50.128 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.128 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.129 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-08-11T15:56:50.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-08-11T15:56:50.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.129 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.130 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-08-11T15:56:50.130 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.130 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.130 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.131 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-08-11T15:56:50.131 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.131 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.132 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-08-11T15:56:50.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-08-11T15:56:50.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.133 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.133 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.133 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.133 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-08-11T15:56:50.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.134 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.134 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-08-11T15:56:50.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-08-11T15:56:50.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.135 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.135 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.135 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.136 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.136 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.136 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-11T15:56:50.136 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.137 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-08-11T15:56:50.137 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-08-11T15:56:50.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.138 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.138 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.138 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.139 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-08-11T15:56:50.139 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.139 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.140 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-08-11T15:56:50.141 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.141 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.141 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.141 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-08-11T15:56:50.141 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-08-11T15:56:50.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.142 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.142 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.142 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.143 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-08-11T15:56:50.143 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.143 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.144 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.144 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-08-11T15:56:50.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-08-11T15:56:50.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.145 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.145 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-08-11T15:56:50.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.146 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.146 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-08-11T15:56:50.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T15:56:50.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.148 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-08-11T15:56:50.148 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-08-11T15:56:50.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:50.149 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:50.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-08-11T15:56:50.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-08-11T15:56:50.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.151 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-11T15:56:50.151 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T15:56:50.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.152 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-08-11T15:56:50.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-08-11T15:56:50.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:50.153 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.154 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-08-11T15:56:50.154 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:50.154 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-08-11T15:56:50.155 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-11T15:56:50.155 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.157 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-08-11T15:56:50.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-08-11T15:56:50.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.158 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.158 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-08-11T15:56:50.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.159 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-11T15:56:50.159 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-11T15:56:50.160 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.160 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.160 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-08-11T15:56:50.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-08-11T15:56:50.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:50.161 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.161 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-08-11T15:56:50.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-08-11T15:56:50.162 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-08-11T15:56:50.162 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.162 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-08-11T15:56:50.162 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-11T15:56:50.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.163 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.163 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-08-11T15:56:50.163 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-08-11T15:56:50.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.164 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.164 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.164 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-08-11T15:56:50.164 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.164 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-08-11T15:56:50.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T15:56:50.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.166 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-08-11T15:56:50.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-08-11T15:56:50.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:50.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.167 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.167 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:50.167 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-08-11T15:56:50.167 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-08-11T15:56:50.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.168 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-11T15:56:50.168 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T15:56:50.168 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.168 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.169 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.169 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.169 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.171 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.171 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.171 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.171 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.171 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.172 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.172 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.172 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.173 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.173 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.174 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.174 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.175 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.175 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.175 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.175 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.176 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.176 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.178 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.178 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.178 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.178 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.178 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.178 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.179 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.179 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.179 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.179 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.180 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.180 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.181 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.181 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.182 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.182 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.182 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-08-11T15:56:50.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-08-11T15:56:50.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:50.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.184 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:50.184 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.184 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.184 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-11T15:56:50.185 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.185 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.185 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.185 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-08-11T15:56:50.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-08-11T15:56:50.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.186 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.187 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-08-11T15:56:50.187 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.187 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.187 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.187 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-08-11T15:56:50.188 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-08-11T15:56:50.188 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-08-11T15:56:50.188 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.188 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-08-11T15:56:50.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.189 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.189 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-08-11T15:56:50.190 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-08-11T15:56:50.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.190 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.191 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-11T15:56:50.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.193 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.193 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:50.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:50.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.194 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.194 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:50.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.195 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-11T15:56:50.195 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.195 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.196 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-08-11T15:56:50.196 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-08-11T15:56:50.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.196 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.196 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:50.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:50.197 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.198 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.198 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-11T15:56:50.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-11T15:56:50.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-08-11T15:56:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-08-11T15:56:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-11T15:56:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.202 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.202 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-08-11T15:56:50.203 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.203 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.203 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.203 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-08-11T15:56:50.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-08-11T15:56:50.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:50.204 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.204 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-08-11T15:56:50.204 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.205 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.205 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.205 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-11T15:56:50.205 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.205 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.206 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.206 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-08-11T15:56:50.206 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-08-11T15:56:50.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.207 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.207 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-08-11T15:56:50.207 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.207 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.208 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.208 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-11T15:56:50.208 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.208 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.208 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.209 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-08-11T15:56:50.209 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-08-11T15:56:50.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.209 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.210 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-08-11T15:56:50.210 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.210 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.210 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.211 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-11T15:56:50.211 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.211 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.212 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-08-11T15:56:50.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-08-11T15:56:50.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.212 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.213 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.213 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.213 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.213 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-11T15:56:50.213 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-11T15:56:50.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.214 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.214 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-08-11T15:56:50.214 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-08-11T15:56:50.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:50.215 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.216 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.216 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.216 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-11T15:56:50.216 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.217 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-08-11T15:56:50.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-08-11T15:56:50.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:50.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-08-11T15:56:50.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.219 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.219 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-11T15:56:50.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.220 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-08-11T15:56:50.220 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-08-11T15:56:50.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.221 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-11T15:56:50.222 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.222 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.222 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-08-11T15:56:50.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-08-11T15:56:50.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.223 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.223 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.224 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.224 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.224 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.224 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-11T15:56:50.224 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.225 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.225 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-08-11T15:56:50.225 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-08-11T15:56:50.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.226 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.226 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.226 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.227 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.227 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.227 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-11T15:56:50.227 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.227 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.228 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-08-11T15:56:50.228 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-08-11T15:56:50.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.228 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.229 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-11T15:56:50.229 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.229 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.229 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.230 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:50.230 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.230 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.231 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-08-11T15:56:50.231 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-08-11T15:56:50.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.231 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.231 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.231 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-11T15:56:50.232 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.232 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.232 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-08-11T15:56:50.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.233 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.233 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-08-11T15:56:50.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-08-11T15:56:50.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.234 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.234 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-11T15:56:50.234 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.235 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.235 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.235 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-08-11T15:56:50.235 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.235 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.236 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-08-11T15:56:50.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-08-11T15:56:50.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.237 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-11T15:56:50.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.238 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:50.238 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.239 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-08-11T15:56:50.239 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-08-11T15:56:50.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.239 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.240 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-11T15:56:50.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.240 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.240 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.240 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-08-11T15:56:50.241 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.241 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.241 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.241 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-08-11T15:56:50.242 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-08-11T15:56:50.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.242 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-11T15:56:50.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.243 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-08-11T15:56:50.243 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.244 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.244 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-08-11T15:56:50.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-08-11T15:56:50.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.245 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.245 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-11T15:56:50.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.246 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.246 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-08-11T15:56:50.246 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.247 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-08-11T15:56:50.247 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-08-11T15:56:50.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.247 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.248 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-11T15:56:50.248 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.248 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.248 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.249 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-08-11T15:56:50.249 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.249 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.250 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-08-11T15:56:50.250 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-08-11T15:56:50.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.250 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-11T15:56:50.251 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.251 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.251 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-08-11T15:56:50.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.252 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-08-11T15:56:50.253 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-08-11T15:56:50.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-11T15:56:50.254 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.254 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.254 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.254 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-08-11T15:56:50.254 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.255 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.255 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-08-11T15:56:50.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-08-11T15:56:50.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.256 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-11T15:56:50.256 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.257 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.257 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.257 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:50.257 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.258 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.258 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-08-11T15:56:50.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-08-11T15:56:50.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.259 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.259 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.1", 2023-08-11T15:56:50.259 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.259 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.260 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-08-11T15:56:50.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.260 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.261 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-08-11T15:56:50.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-08-11T15:56:50.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.261 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.262 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-11T15:56:50.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.262 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.262 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-08-11T15:56:50.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.263 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.263 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.263 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-08-11T15:56:50.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-08-11T15:56:50.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.264 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.264 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-11T15:56:50.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.265 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-08-11T15:56:50.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.266 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.266 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-08-11T15:56:50.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-08-11T15:56:50.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.267 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.267 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-11T15:56:50.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.268 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.268 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-08-11T15:56:50.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.269 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-08-11T15:56:50.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-08-11T15:56:50.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.269 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-08-11T15:56:50.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.271 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-08-11T15:56:50.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-08-11T15:56:50.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:50.273 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.273 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.273 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.273 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-11T15:56:50.273 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.274 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.274 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-08-11T15:56:50.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-08-11T15:56:50.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.275 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-08-11T15:56:50.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.276 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.276 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.276 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-11T15:56:50.276 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.277 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-08-11T15:56:50.277 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-08-11T15:56:50.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.278 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.278 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-08-11T15:56:50.278 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.278 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.279 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.279 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-11T15:56:50.279 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.279 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.280 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-08-11T15:56:50.280 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-08-11T15:56:50.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.280 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.280 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-08-11T15:56:50.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.281 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.281 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-08-11T15:56:50.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.282 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.282 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-08-11T15:56:50.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-08-11T15:56:50.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.283 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-11T15:56:50.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.285 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.285 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-08-11T15:56:50.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-08-11T15:56:50.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.286 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:50.286 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.287 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-08-11T15:56:50.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.287 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.288 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-08-11T15:56:50.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-08-11T15:56:50.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.288 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.289 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:50.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.289 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.289 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-08-11T15:56:50.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.291 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.291 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.292 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.292 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.293 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.293 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.293 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.293 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.293 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.294 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.295 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-08-11T15:56:50.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-08-11T15:56:50.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.296 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.296 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-11T15:56:50.297 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.297 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.297 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.299 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.299 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.299 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-11T15:56:50.299 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.300 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.300 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.300 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.301 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.301 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.302 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.302 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.302 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.302 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.303 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.303 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.303 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.303 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.304 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.304 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.305 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.306 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.306 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.306 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.307 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.307 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-11T15:56:50.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-11T15:56:50.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.307 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.308 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.308 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.308 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.308 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-11T15:56:50.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.310 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-11T15:56:50.310 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-11T15:56:50.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.310 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.311 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-08-11T15:56:50.311 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.311 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.311 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.311 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-11T15:56:50.312 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.312 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-08-11T15:56:50.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-08-11T15:56:50.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.313 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.313 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-08-11T15:56:50.314 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.315 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.315 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-08-11T15:56:50.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-08-11T15:56:50.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.317 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.317 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-11T15:56:50.317 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.318 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-08-11T15:56:50.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-08-11T15:56:50.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.319 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.319 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.320 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-08-11T15:56:50.320 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.320 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.321 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-08-11T15:56:50.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-08-11T15:56:50.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.321 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.322 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-08-11T15:56:50.322 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.322 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.322 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.322 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-08-11T15:56:50.323 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-08-11T15:56:50.323 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-08-11T15:56:50.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.323 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.324 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-08-11T15:56:50.324 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-08-11T15:56:50.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.324 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.325 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-11T15:56:50.325 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.325 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.325 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.326 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-11T15:56:50.326 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.326 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.327 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-08-11T15:56:50.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-08-11T15:56:50.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.327 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.328 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.328 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.328 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.328 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.328 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-11T15:56:50.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.329 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-08-11T15:56:50.329 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-08-11T15:56:50.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.330 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.330 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.331 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.331 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.331 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-11T15:56:50.331 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.332 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.332 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-08-11T15:56:50.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-08-11T15:56:50.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.333 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.333 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.333 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.334 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-11T15:56:50.334 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.334 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.334 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.335 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-08-11T15:56:50.335 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-08-11T15:56:50.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.335 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-11T15:56:50.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.337 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-08-11T15:56:50.338 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-08-11T15:56:50.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.338 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.338 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.339 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.339 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.339 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-11T15:56:50.339 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.340 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.340 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-08-11T15:56:50.340 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-08-11T15:56:50.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-08-11T15:56:50.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-08-11T15:56:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-08-11T15:56:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-08-11T15:56:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-11T15:56:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-08-11T15:56:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-08-11T15:56:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-11T15:56:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-08-11T15:56:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-08-11T15:56:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-11T15:56:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-08-11T15:56:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-08-11T15:56:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-08-11T15:56:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-08-11T15:56:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-08-11T15:56:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-11T15:56:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-08-11T15:56:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-08-11T15:56:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-11T15:56:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-08-11T15:56:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-08-11T15:56:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-08-11T15:56:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-08-11T15:56:50.376 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.376 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.376 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-11T15:56:50.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.377 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.377 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-08-11T15:56:50.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-08-11T15:56:50.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.378 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-08-11T15:56:50.378 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.379 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.379 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.379 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-11T15:56:50.379 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.380 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-08-11T15:56:50.381 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.381 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.382 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.382 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-11T15:56:50.382 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-11T15:56:50.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.382 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.383 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.383 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.384 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.384 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.385 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-08-11T15:56:50.385 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-11T15:56:50.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.385 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.386 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.386 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.386 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.386 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.386 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.387 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.387 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.387 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-11T15:56:50.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-11T15:56:50.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.388 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.388 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.388 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.389 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.389 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.389 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.390 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.390 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-08-11T15:56:50.390 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-08-11T15:56:50.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.391 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.391 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.391 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.391 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.392 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.392 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.392 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.392 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.393 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-11T15:56:50.393 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-11T15:56:50.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.393 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.394 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.394 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.394 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.394 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.395 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.395 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.395 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.396 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-08-11T15:56:50.396 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-11T15:56:50.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.396 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.397 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.397 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.397 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.397 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.398 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.398 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-11T15:56:50.399 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-11T15:56:50.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.400 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.400 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.400 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.400 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.400 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.401 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.401 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-11T15:56:50.401 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-11T15:56:50.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.402 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.402 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.402 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.403 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.403 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.403 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.403 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.404 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.404 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-11T15:56:50.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-11T15:56:50.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.405 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.405 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.405 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.406 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.406 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.406 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.406 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.406 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.406 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.407 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-11T15:56:50.407 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-11T15:56:50.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.407 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.408 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.408 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.409 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.409 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.409 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.410 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-11T15:56:50.410 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-11T15:56:50.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.410 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.411 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.411 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.411 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.411 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.411 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.412 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.412 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.412 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-11T15:56:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-11T15:56:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.413 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.414 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-11T15:56:50.415 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-11T15:56:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.416 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.417 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.417 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.417 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.417 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.417 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.417 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.418 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-11T15:56:50.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-11T15:56:50.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.418 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.419 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.419 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.419 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.419 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.420 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.420 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.420 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.421 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-08-11T15:56:50.421 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-11T15:56:50.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.421 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.421 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.422 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:50.422 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.422 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.422 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.422 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.423 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.423 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.423 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.423 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-11T15:56:50.424 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-11T15:56:50.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.424 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.424 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.425 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.425 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.425 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.425 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.425 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.426 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.426 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-08-11T15:56:50.426 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-08-11T15:56:50.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.427 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.427 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.427 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.427 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.428 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.428 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.428 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.428 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.429 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-08-11T15:56:50.429 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-11T15:56:50.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.429 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.430 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:50.430 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.430 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.430 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.431 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.431 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.431 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.432 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-11T15:56:50.432 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-11T15:56:50.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.432 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.433 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.433 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.433 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.433 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.433 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.434 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.434 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-11T15:56:50.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-11T15:56:50.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.435 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.436 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.436 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.436 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.436 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.437 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.437 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-11T15:56:50.437 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-11T15:56:50.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.438 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.438 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.439 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.439 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.439 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.439 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.439 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.440 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-11T15:56:50.440 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-11T15:56:50.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.440 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.441 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.441 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.441 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.442 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.442 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.442 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.442 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.443 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-08-11T15:56:50.443 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-11T15:56:50.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.443 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.444 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.444 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.444 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.444 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.444 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.445 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.445 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.445 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-11T15:56:50.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-11T15:56:50.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.446 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.446 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.446 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.447 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.447 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.447 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.447 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.447 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.448 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-11T15:56:50.448 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-11T15:56:50.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.449 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.450 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.450 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.450 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.450 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.451 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-11T15:56:50.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-11T15:56:50.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.452 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.452 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.452 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.452 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.453 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.453 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.453 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-11T15:56:50.454 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-11T15:56:50.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.454 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.454 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.454 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.455 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.455 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.455 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.455 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.455 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.456 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.456 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-11T15:56:50.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-11T15:56:50.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.457 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.457 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.457 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.458 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.458 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.458 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.458 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.459 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-11T15:56:50.459 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-11T15:56:50.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.459 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.460 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.460 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.462 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-11T15:56:50.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-11T15:56:50.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-11T15:56:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-11T15:56:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.466 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.467 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.467 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-08-11T15:56:50.467 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-11T15:56:50.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.468 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.468 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.468 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.469 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.469 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.469 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.469 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.470 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-11T15:56:50.470 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-11T15:56:50.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.470 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-11T15:56:50.471 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.471 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.471 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.472 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.472 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.472 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.473 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-11T15:56:50.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-11T15:56:50.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.473 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.473 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.474 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.474 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.474 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.474 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.474 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.475 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.475 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-11T15:56:50.475 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-11T15:56:50.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.476 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.476 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-08-11T15:56:50.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.477 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.477 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.477 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.478 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-11T15:56:50.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-11T15:56:50.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.479 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.479 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.479 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-08-11T15:56:50.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.480 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.480 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.480 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.480 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.481 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-08-11T15:56:50.481 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-11T15:56:50.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.481 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.482 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.482 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.482 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.482 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.482 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.483 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.483 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.483 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.483 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-11T15:56:50.484 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-11T15:56:50.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.484 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.484 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-08-11T15:56:50.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.485 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.485 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.485 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.485 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-08-11T15:56:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-11T15:56:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.487 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.487 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.488 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.488 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.488 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.488 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.488 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.489 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-08-11T15:56:50.489 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-11T15:56:50.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.489 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.491 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.491 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.491 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.492 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-11T15:56:50.492 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-11T15:56:50.492 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.492 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.492 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-11T15:56:50.495 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-11T15:56:50.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.495 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.495 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.496 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.496 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.496 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.496 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.496 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-11T15:56:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-11T15:56:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.498 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.498 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:50.498 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.498 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.499 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.499 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.499 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.499 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-08-11T15:56:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-08-11T15:56:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-08-11T15:56:50.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-08-11T15:56:50.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.503 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.504 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.504 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.504 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.504 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.504 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-08-11T15:56:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-08-11T15:56:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.507 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-08-11T15:56:50.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.507 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.508 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-08-11T15:56:50.508 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-08-11T15:56:50.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-08-11T15:56:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.510 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.510 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.510 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.510 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-08-11T15:56:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-08-11T15:56:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.512 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.512 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.512 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.512 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.515 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.515 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.515 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.515 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.515 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.517 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.517 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.517 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.519 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.519 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.520 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.520 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.520 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.521 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.521 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.521 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.521 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.521 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.522 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.522 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.522 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.522 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.522 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.523 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.523 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.523 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.523 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.525 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.525 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.525 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.525 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.526 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.526 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.526 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.526 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.526 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.527 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-08-11T15:56:50.527 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-08-11T15:56:50.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.527 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.528 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.528 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.528 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-08-11T15:56:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-08-11T15:56:50.530 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-08-11T15:56:50.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.530 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.530 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.531 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.531 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.531 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.531 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.532 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.532 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:50.532 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-08-11T15:56:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.534 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.534 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.534 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.534 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-08-11T15:56:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.537 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.537 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.537 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-08-11T15:56:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-08-11T15:56:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.539 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.539 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.539 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.539 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.539 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.540 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.540 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.540 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-08-11T15:56:50.541 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-08-11T15:56:50.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.541 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.541 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-08-11T15:56:50.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.542 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.542 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.542 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.542 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-08-11T15:56:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-08-11T15:56:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.544 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.544 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-08-11T15:56:50.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.544 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.545 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.545 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.545 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-08-11T15:56:50.545 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.545 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.546 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.546 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-08-11T15:56:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-08-11T15:56:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-08-11T15:56:50.548 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-08-11T15:56:50.548 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-08-11T15:56:50.548 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.548 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.548 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.549 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.549 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-08-11T15:56:50.549 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-08-11T15:56:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-08-11T15:56:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-08-11T15:56:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-08-11T15:56:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-08-11T15:56:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-08-11T15:56:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-08-11T15:56:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-08-11T15:56:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-11T15:56:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-08-11T15:56:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-08-11T15:56:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-08-11T15:56:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-11T15:56:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-08-11T15:56:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-08-11T15:56:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-08-11T15:56:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-08-11T15:56:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-08-11T15:56:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-08-11T15:56:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-08-11T15:56:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-11T15:56:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-08-11T15:56:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-08-11T15:56:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-11T15:56:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-08-11T15:56:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-08-11T15:56:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-08-11T15:56:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-08-11T15:56:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-11T15:56:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-08-11T15:56:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-08-11T15:56:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-08-11T15:56:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-08-11T15:56:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-08-11T15:56:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-08-11T15:56:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-08-11T15:56:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-08-11T15:56:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-08-11T15:56:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-08-11T15:56:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-08-11T15:56:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-08-11T15:56:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-08-11T15:56:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-08-11T15:56:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-11T15:56:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-08-11T15:56:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-08-11T15:56:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-08-11T15:56:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-11T15:56:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-08-11T15:56:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-08-11T15:56:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-08-11T15:56:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-08-11T15:56:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-08-11T15:56:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-11T15:56:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-08-11T15:56:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-08-11T15:56:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-08-11T15:56:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-08-11T15:56:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-08-11T15:56:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-08-11T15:56:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-08-11T15:56:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-08-11T15:56:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-08-11T15:56:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-08-11T15:56:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T15:56:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-08-11T15:56:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-08-11T15:56:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-08-11T15:56:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-11T15:56:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-08-11T15:56:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-08-11T15:56:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-08-11T15:56:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-08-11T15:56:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-08-11T15:56:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-08-11T15:56:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T15:56:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-08-11T15:56:50.655 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.655 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.655 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.655 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.655 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.656 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.656 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.656 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.657 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.657 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.657 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.657 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.657 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.658 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.658 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.658 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.658 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.658 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.659 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.659 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.659 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.659 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.660 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.660 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.660 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.661 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.661 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.661 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.661 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.662 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.662 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.662 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-08-11T15:56:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2023-08-11T15:56:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2023-08-11T15:56:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2023-08-11T15:56:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2023-08-11T15:56:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-11T15:56:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2023-08-11T15:56:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2023-08-11T15:56:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2023-08-11T15:56:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-11T15:56:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.700 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.707 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.707 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-08-11T15:56:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.710 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.710 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-08-11T15:56:50.710 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-08-11T15:56:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-08-11T15:56:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-08-11T15:56:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-11T15:56:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-08-11T15:56:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-08-11T15:56:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-08-11T15:56:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-11T15:56:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-08-11T15:56:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-08-11T15:56:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-08-11T15:56:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-08-11T15:56:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-08-11T15:56:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-08-11T15:56:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.734 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-08-11T15:56:50.734 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.734 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.734 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.734 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-08-11T15:56:50.735 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.735 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.735 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-08-11T15:56:50.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-08-11T15:56:50.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.736 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.736 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.736 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.737 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.737 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.737 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-08-11T15:56:50.737 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.738 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-08-11T15:56:50.738 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-08-11T15:56:50.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.739 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.739 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-08-11T15:56:50.739 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.739 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.740 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-11T15:56:50.740 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-11T15:56:50.740 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.740 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-08-11T15:56:50.742 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.742 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.742 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.742 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.742 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.743 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.743 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.743 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.743 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.743 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.744 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.744 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.744 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.744 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.744 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.745 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.745 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.745 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.745 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.745 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.746 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.746 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.747 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.747 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.747 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.747 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.748 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.748 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.748 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.749 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.749 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.749 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.750 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.750 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.750 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.750 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.751 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.751 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.751 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.751 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.751 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.752 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.752 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.752 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.752 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.753 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.753 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.753 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.753 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.754 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.754 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.754 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.755 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.755 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.755 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.756 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-08-11T15:56:50.756 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.757 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.757 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.757 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.758 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.758 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.758 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.758 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.758 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.759 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.759 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.759 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.759 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.759 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.760 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.760 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.760 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.760 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.760 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.761 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.761 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.761 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.762 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.762 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.762 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.762 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.763 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.763 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.763 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.763 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.764 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.764 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.764 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.765 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.765 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.765 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.765 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.765 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.766 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.766 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.766 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.766 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.766 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.766 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.767 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.767 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.767 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.767 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.767 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.768 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.768 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.768 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.769 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.769 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.769 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.769 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.770 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.770 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.770 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.771 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-08-11T15:56:50.771 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.772 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.772 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.772 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.773 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.773 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.773 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.773 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.774 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.774 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.774 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.774 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.774 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.775 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.775 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.775 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.775 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.776 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.776 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.776 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.777 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.777 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.777 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.777 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.777 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.778 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.778 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.778 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.779 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.779 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.779 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.780 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.780 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.780 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.780 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.780 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.781 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.781 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.781 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.781 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.781 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.782 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.782 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.782 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.782 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.783 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.783 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.783 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.784 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.784 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.784 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.784 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.784 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.785 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.785 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.786 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-08-11T15:56:50.786 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.786 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.787 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.787 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-08-11T15:56:50.787 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-08-11T15:56:50.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.787 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.788 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.788 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.788 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.788 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.788 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-11T15:56:50.789 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.789 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.789 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.790 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.790 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.790 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.791 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.791 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.791 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.791 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.791 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.792 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.792 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.792 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.792 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.792 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.793 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.793 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.793 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.794 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.794 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.794 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.795 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.795 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.795 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.795 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.796 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.796 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.797 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.797 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.797 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.797 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.798 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.798 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.798 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.798 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.798 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.799 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.799 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.799 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.799 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.799 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.800 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.800 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.800 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.801 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.801 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.801 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.802 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.802 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.802 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.802 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.802 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.803 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.803 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-08-11T15:56:50.803 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-08-11T15:56:50.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.804 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.804 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.804 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.805 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.805 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-08-11T15:56:50.805 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-08-11T15:56:50.805 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-08-11T15:56:50.806 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.806 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-08-11T15:56:50.806 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.806 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.807 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-08-11T15:56:50.807 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.808 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.808 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.808 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.808 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.809 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.809 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.809 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.809 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.810 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.810 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.810 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.810 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.810 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.811 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.811 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.811 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.811 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.811 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.812 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.812 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.812 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.813 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.813 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.813 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.814 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.814 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.814 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.814 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.815 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.815 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.815 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.816 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.816 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.816 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.816 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.816 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.817 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.817 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.817 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.817 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.817 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.817 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.818 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.818 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.818 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.818 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.819 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.819 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.819 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.820 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.820 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.820 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.821 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.821 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.821 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.822 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.822 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-08-11T15:56:50.823 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.823 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.823 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.823 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.823 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.824 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.824 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.824 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.825 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.825 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.825 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.825 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.825 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.826 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.826 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.826 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.826 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.826 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.827 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.827 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.827 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.828 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.828 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.828 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.829 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.829 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.829 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.829 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.830 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.830 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.830 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.831 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.831 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.831 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.831 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.832 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.832 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.832 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.832 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.832 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.833 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.833 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.833 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.833 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.833 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.834 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.834 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.834 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.834 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.835 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.835 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.835 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.836 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.836 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.836 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.836 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.837 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.837 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-08-11T15:56:50.838 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.838 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.838 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.838 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.838 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.839 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.839 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.839 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.840 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.840 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.840 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.840 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.840 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.841 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.841 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.841 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.841 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.841 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.842 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.842 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.842 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.843 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.843 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.843 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.843 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.844 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.844 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.844 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.844 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.845 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.845 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.845 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.846 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.846 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.846 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.847 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.847 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.847 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.847 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.847 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.848 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.848 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.848 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.848 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.848 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.849 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.849 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.849 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.850 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.850 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.850 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.850 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.851 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.851 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.851 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.851 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.852 INFO:teuthology.orchestra.run.smithi032.stdout: "noautoscale": { 2023-08-11T15:56:50.852 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "noautoscale", 2023-08-11T15:56:50.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.852 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.853 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.853 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.853 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.853 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.854 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "global autoscale flag", 2023-08-11T15:56:50.854 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-11T15:56:50.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.854 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.855 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:50.855 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:50.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.855 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.856 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:50.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.856 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.856 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.856 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.857 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.857 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.857 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-08-11T15:56:50.858 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-08-11T15:56:50.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:50.858 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.858 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-08-11T15:56:50.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-08-11T15:56:50.859 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.859 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.859 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-08-11T15:56:50.859 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-11T15:56:50.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.860 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.861 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-08-11T15:56:50.861 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.861 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.861 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.861 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-08-11T15:56:50.862 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-08-11T15:56:50.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.862 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.862 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.863 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.863 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.863 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-11T15:56:50.863 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.864 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.864 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-08-11T15:56:50.864 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-08-11T15:56:50.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.865 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.865 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.865 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.866 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.866 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.866 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.866 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.866 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.867 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.867 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.867 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.868 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.868 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.868 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.868 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.869 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.869 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.869 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.869 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.869 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.870 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.870 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.870 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.870 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.870 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.870 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.871 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.871 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.871 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.872 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.872 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.872 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.872 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.873 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.873 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.873 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.873 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.873 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.874 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.874 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.875 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.875 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.875 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.875 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.876 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.876 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.876 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.876 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.876 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.877 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.877 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.877 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.877 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.877 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.880 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.880 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.880 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.881 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-08-11T15:56:50.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-08-11T15:56:50.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.881 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.882 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.882 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-08-11T15:56:50.882 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.882 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-08-11T15:56:50.883 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.883 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.883 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-08-11T15:56:50.884 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-08-11T15:56:50.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:50.884 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.884 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-08-11T15:56:50.885 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.885 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.885 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.885 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-08-11T15:56:50.885 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.886 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.887 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-08-11T15:56:50.887 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.887 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.887 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-08-11T15:56:50.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-08-11T15:56:50.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.888 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.888 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.889 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.890 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.890 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.890 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-08-11T15:56:50.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-08-11T15:56:50.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.891 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.891 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.891 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.892 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.892 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.892 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-11T15:56:50.892 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-11T15:56:50.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.893 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.893 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.893 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.894 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.894 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.894 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.894 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.894 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.895 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.895 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.895 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.895 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.895 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.896 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.896 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.896 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.896 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.897 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.897 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.898 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.898 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.898 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.899 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.899 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.900 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.900 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.901 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.901 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.901 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.901 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.902 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.902 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.902 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.902 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.902 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.903 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.903 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.903 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.903 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.904 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.904 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.904 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.905 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.905 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.905 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.905 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.906 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.907 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-08-11T15:56:50.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-08-11T15:56:50.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.908 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.908 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.909 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.909 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.909 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-11T15:56:50.909 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-11T15:56:50.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.910 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.910 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-08-11T15:56:50.910 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.911 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.911 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.911 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.911 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.912 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.912 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-08-11T15:56:50.912 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-08-11T15:56:50.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:50.913 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.913 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-08-11T15:56:50.913 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.913 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.914 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.914 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.914 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.914 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.914 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.915 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:50.915 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:50.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.915 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.916 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-08-11T15:56:50.916 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.916 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.916 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.916 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-11T15:56:50.917 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.917 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.917 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:50.918 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:50.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.918 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.918 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-08-11T15:56:50.919 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.919 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.919 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.919 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-11T15:56:50.919 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.920 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.920 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-08-11T15:56:50.920 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-08-11T15:56:50.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.921 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.921 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-08-11T15:56:50.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.921 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.922 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.922 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.922 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.922 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.923 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-08-11T15:56:50.923 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-08-11T15:56:50.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.923 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.924 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-08-11T15:56:50.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.924 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.924 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.924 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-08-11T15:56:50.925 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-08-11T15:56:50.925 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.925 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.925 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.925 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-08-11T15:56:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-08-11T15:56:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-08-11T15:56:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-08-11T15:56:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-08-11T15:56:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-08-11T15:56:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.930 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.930 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.930 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.930 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.931 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.931 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.931 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.932 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.932 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.932 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.932 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.932 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.933 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.933 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.933 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.933 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.933 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.934 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.934 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.935 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.936 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.936 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.936 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.936 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.937 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.937 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.937 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.938 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.938 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.938 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.938 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.939 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.939 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.939 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.939 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.939 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.939 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.940 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.940 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.940 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.941 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.941 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.942 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.942 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.942 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.943 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.943 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-08-11T15:56:50.944 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-08-11T15:56:50.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:50.944 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-08-11T15:56:50.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.945 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.945 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.945 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.945 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.946 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-08-11T15:56:50.946 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-08-11T15:56:50.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.948 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.948 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.948 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.949 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.949 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-08-11T15:56:50.949 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-08-11T15:56:50.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.950 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.950 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.950 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.950 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.950 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.951 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.951 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.951 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.952 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-08-11T15:56:50.952 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.953 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.953 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.953 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-08-11T15:56:50.953 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-08-11T15:56:50.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.954 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.954 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:50.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.954 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.955 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.955 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.955 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.955 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.956 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-08-11T15:56:50.956 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-08-11T15:56:50.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.956 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.957 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.957 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.957 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.957 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.958 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.959 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.959 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.959 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.959 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.960 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.960 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.960 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.960 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.960 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.961 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.961 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.961 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.961 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.961 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.962 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.962 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.963 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.963 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.964 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.964 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.964 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.964 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.965 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.965 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.965 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.966 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.966 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.967 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.967 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.967 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.967 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.967 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.968 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.968 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.968 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.968 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.968 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.969 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.969 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.971 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.971 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.971 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.972 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.972 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-08-11T15:56:50.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-08-11T15:56:50.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.973 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.973 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.973 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.974 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.975 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-08-11T15:56:50.975 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-08-11T15:56:50.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.976 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.976 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.976 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.976 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.977 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.977 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.977 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-08-11T15:56:50.978 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.978 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.979 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.979 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.979 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.980 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.980 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.980 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.981 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.981 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.981 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.981 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.981 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.982 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.982 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.982 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.982 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.982 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.983 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:50.983 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:50.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.983 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.984 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.984 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.984 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.985 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.986 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:50.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:50.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:50.987 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:50.987 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:50.987 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:50.988 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:50.988 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:50.988 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:50.988 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:50.988 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:50.989 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.990 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:50.990 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:50.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:50.991 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.991 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:50.992 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.992 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.993 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:50.993 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.993 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:50.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:50.994 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-08-11T15:56:50.995 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:50.995 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:50.995 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:50.995 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-08-11T15:56:50.995 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-08-11T15:56:50.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:50.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:50.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-08-11T15:56:50.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.997 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:50.997 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:50.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-11T15:56:50.997 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:50.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:50.998 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:50.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:50.998 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:50.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:50.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:50.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:50.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:50.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:50.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:50.999 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.000 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.000 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.000 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.000 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.001 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.001 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.001 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.001 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.001 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.002 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.002 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.002 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.004 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.004 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.004 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.004 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.005 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.005 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.005 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.006 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.008 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.008 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.008 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.009 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.009 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.009 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.010 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.010 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.010 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.011 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.011 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.012 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.013 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.013 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.015 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.015 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.016 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.016 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-08-11T15:56:51.016 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-08-11T15:56:51.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.017 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-08-11T15:56:51.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.019 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.019 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-11T15:56:51.019 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-08-11T15:56:51.023 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.023 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.023 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.024 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.024 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.026 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.027 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.027 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.028 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.028 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.028 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.029 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.029 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.030 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.030 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.030 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.031 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.032 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.033 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.033 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.034 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.035 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.035 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.035 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.036 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.037 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.037 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.038 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.038 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.039 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.040 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.040 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.040 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.041 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.041 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.042 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.042 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.042 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.043 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.043 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.044 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.045 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.045 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.046 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.046 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.048 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-08-11T15:56:51.048 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-08-11T15:56:51.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.048 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.049 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.049 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.050 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.050 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.051 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-08-11T15:56:51.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-08-11T15:56:51.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.051 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-08-11T15:56:51.052 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.052 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.052 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.053 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.053 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-08-11T15:56:51.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-08-11T15:56:51.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.054 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.054 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.055 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.055 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.055 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.055 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.056 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.057 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-08-11T15:56:51.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-08-11T15:56:51.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:51.057 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.058 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.058 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.058 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.058 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.058 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.059 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.060 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-08-11T15:56:51.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-08-11T15:56:51.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-08-11T15:56:51.060 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.061 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.062 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.062 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.062 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.064 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-08-11T15:56:51.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-08-11T15:56:51.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.065 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.065 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.066 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.067 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.067 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.068 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.069 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-08-11T15:56:51.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-08-11T15:56:51.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.070 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.072 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.072 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.073 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.073 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.074 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.075 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-08-11T15:56:51.075 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-08-11T15:56:51.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.075 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.076 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:51.076 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.077 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.077 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.077 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.078 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-08-11T15:56:51.078 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-08-11T15:56:51.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:51.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.079 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.079 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-08-11T15:56:51.079 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-08-11T15:56:51.079 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.080 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.080 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.080 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.081 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-08-11T15:56:51.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-08-11T15:56:51.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.081 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.082 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.082 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.082 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.083 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.083 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.084 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-08-11T15:56:51.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-08-11T15:56:51.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.084 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.086 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.087 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.087 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-08-11T15:56:51.088 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-08-11T15:56:51.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.088 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.089 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.089 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.089 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.090 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.090 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-08-11T15:56:51.094 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.094 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.094 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.094 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-08-11T15:56:51.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-08-11T15:56:51.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.095 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.095 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.096 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-08-11T15:56:51.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.097 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-08-11T15:56:51.098 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-08-11T15:56:51.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.098 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.099 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.099 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.099 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-08-11T15:56:51.101 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.101 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.101 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.102 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.102 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.102 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.103 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.103 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.103 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.104 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.104 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.104 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.104 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.105 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.105 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.105 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.106 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.106 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.107 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.107 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.107 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.107 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.108 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.108 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.109 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.109 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.110 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.110 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.111 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.111 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.111 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.111 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.112 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.112 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.112 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.112 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.112 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.113 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.113 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.114 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.114 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.115 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.115 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.115 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.116 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.116 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-08-11T15:56:51.117 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.118 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.118 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.118 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.119 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.120 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.120 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.120 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.134 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.135 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.135 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.135 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.135 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.135 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.136 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.137 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-08-11T15:56:51.137 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.137 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.138 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.138 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.139 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.140 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.140 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.140 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.140 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.141 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.141 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.141 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.142 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.143 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.144 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.144 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.144 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.144 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.145 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.145 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.145 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.147 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.148 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.151 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-08-11T15:56:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.153 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.154 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-08-11T15:56:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-08-11T15:56:51.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-11T15:56:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.158 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:51.159 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-08-11T15:56:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.160 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.161 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.162 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.163 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.164 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-08-11T15:56:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-08-11T15:56:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-08-11T15:56:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-11T15:56:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-08-11T15:56:51.179 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-08-11T15:56:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:51.180 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-11T15:56:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.181 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-08-11T15:56:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-08-11T15:56:51.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-08-11T15:56:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-11T15:56:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-08-11T15:56:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-08-11T15:56:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.186 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-11T15:56:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.187 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-08-11T15:56:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-08-11T15:56:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.188 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-11T15:56:51.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-08-11T15:56:51.190 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-08-11T15:56:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-08-11T15:56:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-08-11T15:56:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-08-11T15:56:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-08-11T15:56:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.196 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-11T15:56:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.198 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-08-11T15:56:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-08-11T15:56:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.200 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:51.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-08-11T15:56:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-08-11T15:56:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.203 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.204 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-08-11T15:56:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-08-11T15:56:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-08-11T15:56:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.206 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-08-11T15:56:51.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-08-11T15:56:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.208 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-08-11T15:56:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-08-11T15:56:51.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.214 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.215 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.216 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.222 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.222 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.222 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.222 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.222 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.223 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.223 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.224 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.224 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.224 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.225 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.225 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.225 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.225 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.226 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.226 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.226 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.226 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.227 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-08-11T15:56:51.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-08-11T15:56:51.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.228 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.228 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.229 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.229 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.229 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.229 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-08-11T15:56:51.230 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-08-11T15:56:51.230 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.230 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.231 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.231 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.231 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.231 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.231 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.232 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-08-11T15:56:51.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-08-11T15:56:51.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.233 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-11T15:56:51.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.234 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.235 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-08-11T15:56:51.236 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.236 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.236 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.236 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.237 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.237 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.238 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.238 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.238 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.238 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.238 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.239 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.239 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.239 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.239 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.239 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.240 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.241 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.242 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.242 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.242 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.243 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.243 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.243 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.244 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.245 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.245 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.245 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.246 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.246 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.246 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.246 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.246 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.247 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.247 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.247 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.247 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.248 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.248 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.248 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.249 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.249 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.249 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.251 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-08-11T15:56:51.251 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.251 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.251 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.252 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.252 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.252 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.253 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.253 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.254 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.254 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.254 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.254 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.254 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.254 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.255 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.255 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.255 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.256 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.256 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.257 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.257 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.257 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.257 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.258 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.258 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.258 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.258 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.258 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.259 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.259 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.259 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.260 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.260 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.260 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.261 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.261 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.261 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.261 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.261 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.262 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.263 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.263 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.263 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.264 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.264 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.264 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.264 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.264 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.265 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-08-11T15:56:51.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-08-11T15:56:51.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:51.266 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.266 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.266 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-08-11T15:56:51.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.267 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-11T15:56:51.267 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.268 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-08-11T15:56:51.268 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-08-11T15:56:51.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:51.269 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.269 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-08-11T15:56:51.269 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-11T15:56:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-08-11T15:56:51.272 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-08-11T15:56:51.272 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-08-11T15:56:51.272 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-08-11T15:56:51.272 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-08-11T15:56:51.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-08-11T15:56:51.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-08-11T15:56:51.273 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.273 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-08-11T15:56:51.274 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.274 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.274 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.274 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.274 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.275 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.275 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-08-11T15:56:51.275 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-08-11T15:56:51.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.276 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.276 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.276 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.276 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.277 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.277 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.277 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.277 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.277 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.278 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.278 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-08-11T15:56:51.278 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-08-11T15:56:51.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-08-11T15:56:51.279 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.279 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-08-11T15:56:51.279 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.279 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.280 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.280 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.280 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.280 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.281 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-08-11T15:56:51.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-08-11T15:56:51.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.281 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.282 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.282 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.282 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.282 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.283 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.283 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.283 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.283 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.283 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-08-11T15:56:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-08-11T15:56:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.286 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.286 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.287 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.288 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-08-11T15:56:51.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-08-11T15:56:51.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.288 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.289 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-08-11T15:56:51.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.289 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.289 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-08-11T15:56:51.289 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-08-11T15:56:51.290 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-08-11T15:56:51.290 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-08-11T15:56:51.290 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-08-11T15:56:51.290 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-08-11T15:56:51.290 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-08-11T15:56:51.291 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.291 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.291 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.291 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.292 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-08-11T15:56:51.292 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-08-11T15:56:51.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-08-11T15:56:51.292 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-08-11T15:56:51.293 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-08-11T15:56:51.293 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.293 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.293 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.294 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.295 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-08-11T15:56:51.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-08-11T15:56:51.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-08-11T15:56:51.296 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.296 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.297 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.297 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.298 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-08-11T15:56:51.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-08-11T15:56:51.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-08-11T15:56:51.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.299 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-08-11T15:56:51.299 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.299 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.300 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.300 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.300 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.301 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.301 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-08-11T15:56:51.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-08-11T15:56:51.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-08-11T15:56:51.302 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-08-11T15:56:51.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-08-11T15:56:51.302 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-11T15:56:51.303 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-08-11T15:56:51.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-08-11T15:56:51.303 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-08-11T15:56:51.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-08-11T15:56:51.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-08-11T15:56:51.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-08-11T15:56:51.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-08-11T15:56:51.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.305 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.305 INFO:teuthology.orchestra.run.smithi032.stdout: "services": { 2023-08-11T15:56:51.306 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard": "https://172.21.15.32:8443/" 2023-08-11T15:56:51.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.306 INFO:teuthology.orchestra.run.smithi032.stdout: "always_on_modules": { 2023-08-11T15:56:51.307 INFO:teuthology.orchestra.run.smithi032.stdout: "octopus": [ 2023-08-11T15:56:51.307 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-08-11T15:56:51.307 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-08-11T15:56:51.308 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-08-11T15:56:51.308 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-08-11T15:56:51.308 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-08-11T15:56:51.308 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-08-11T15:56:51.309 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-08-11T15:56:51.309 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-08-11T15:56:51.309 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-08-11T15:56:51.309 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-08-11T15:56:51.309 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.310 INFO:teuthology.orchestra.run.smithi032.stdout: "pacific": [ 2023-08-11T15:56:51.310 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-08-11T15:56:51.310 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-08-11T15:56:51.310 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-08-11T15:56:51.310 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-08-11T15:56:51.311 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-08-11T15:56:51.311 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-08-11T15:56:51.311 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-08-11T15:56:51.311 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-08-11T15:56:51.311 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-08-11T15:56:51.312 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-08-11T15:56:51.312 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.312 INFO:teuthology.orchestra.run.smithi032.stdout: "quincy": [ 2023-08-11T15:56:51.312 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-08-11T15:56:51.312 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-08-11T15:56:51.313 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-08-11T15:56:51.313 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-08-11T15:56:51.313 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-08-11T15:56:51.313 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-08-11T15:56:51.313 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-08-11T15:56:51.314 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-08-11T15:56:51.314 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-08-11T15:56:51.314 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-08-11T15:56:51.314 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-08-11T15:56:51.314 INFO:teuthology.orchestra.run.smithi032.stdout: "reef": [ 2023-08-11T15:56:51.314 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-08-11T15:56:51.315 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-08-11T15:56:51.315 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-08-11T15:56:51.315 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-08-11T15:56:51.315 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-08-11T15:56:51.315 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-08-11T15:56:51.316 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-08-11T15:56:51.316 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-08-11T15:56:51.316 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-08-11T15:56:51.316 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-08-11T15:56:51.316 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.317 INFO:teuthology.orchestra.run.smithi032.stdout: "last_failure_osd_epoch": 93, 2023-08-11T15:56:51.317 INFO:teuthology.orchestra.run.smithi032.stdout: "active_clients": [ 2023-08-11T15:56:51.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-08-11T15:56:51.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-08-11T15:56:51.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-08-11T15:56:51.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-08-11T15:56:51.318 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3905825069 2023-08-11T15:56:51.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.319 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.319 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "libcephsqlite", 2023-08-11T15:56:51.320 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-08-11T15:56:51.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-08-11T15:56:51.320 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-08-11T15:56:51.320 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1707255923 2023-08-11T15:56:51.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-08-11T15:56:51.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-08-11T15:56:51.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-08-11T15:56:51.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-08-11T15:56:51.322 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 926480522 2023-08-11T15:56:51.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.323 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.323 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-08-11T15:56:51.323 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-08-11T15:56:51.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-08-11T15:56:51.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-08-11T15:56:51.324 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 915474492 2023-08-11T15:56:51.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-08-11T15:56:51.325 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-08-11T15:56:51.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-08-11T15:56:51.326 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-08-11T15:56:51.326 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1444453454 2023-08-11T15:56:51.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:51.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-08-11T15:56:51.327 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-08-11T15:56:51.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-08-11T15:56:51.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-08-11T15:56:51.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-08-11T15:56:51.328 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3347254290 2023-08-11T15:56:51.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.329 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:51.329 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-08-11T15:56:51.329 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:56:51.331 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:47 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3916601098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:56:51.331 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:47 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3444855198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:56:51.331 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:48 smithi192 ceph-mon[130806]: pgmap v320: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-08-11T15:56:51.332 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:48 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1631103243' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-11T15:56:51.332 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:50 smithi192 ceph-mon[130806]: pgmap v321: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:51.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:47 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3916601098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:56:51.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:47 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3444855198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:56:51.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:48 smithi134 ceph-mon[130447]: pgmap v320: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-08-11T15:56:51.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:48 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1631103243' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-11T15:56:51.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:50 smithi134 ceph-mon[130447]: pgmap v321: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f90e40a0180 msgr2=0x7f90e40a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f90e40a0180 0x7f90e40a2640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f90f4005950 tx=0x7f90f4009310 comp rx=0 tx=0).stop 2023-08-11T15:56:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f81068a0 msgr2=0x7f90f8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:56:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f81068a0 0x7f90f8198920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b6c0 tx=0x7f90f0009500 comp rx=0 tx=0).stop 2023-08-11T15:56:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 shutdown_connections 2023-08-11T15:56:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f90e40a0180 0x7f90e40a2640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90f81068a0 0x7f90f8198920 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f8100890 0x7f90f819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 --2- 172.21.15.32:0/1631103243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90f8101250 0x7f90f819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:56:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 >> 172.21.15.32:0/1631103243 conn(0x7f90f80fc340 msgr2=0x7f90f810bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:56:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 shutdown_connections 2023-08-11T15:56:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:56:47.583+0000 7f90ffdff700 1 -- 172.21.15.32:0/1631103243 wait complete. 2023-08-11T15:56:51.345 DEBUG:tasks.check_counter:Getting stats from a 2023-08-11T15:56:51.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.a perf dump 2023-08-11T15:56:51.346 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:47 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3916601098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:56:51.347 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:47 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3444855198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:56:51.347 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:48 smithi032 ceph-mon[97103]: pgmap v320: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-08-11T15:56:51.347 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:48 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1631103243' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-11T15:56:51.347 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:50 smithi032 ceph-mon[97103]: pgmap v321: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:52.091 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:56:52.621 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:52 smithi032 ceph-mon[97103]: pgmap v322: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:52.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:52 smithi134 ceph-mon[130447]: pgmap v322: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:52.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:52 smithi192 ceph-mon[130806]: pgmap v322: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:53.561 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:56:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:56:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2023-08-11T15:56:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2023-08-11T15:56:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2023-08-11T15:56:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2023-08-11T15:56:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-08-11T15:56:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-08-11T15:56:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.004516508, 2023-08-11T15:56:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2023-08-11T15:56:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001150552, 2023-08-11T15:56:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:56:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T15:56:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T15:56:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:56:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 5, 2023-08-11T15:56:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 57, 2023-08-11T15:56:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 3104, 2023-08-11T15:56:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 86192, 2023-08-11T15:56:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 3, 2023-08-11T15:56:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 2, 2023-08-11T15:56:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.007397718, 2023-08-11T15:56:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.004380835, 2023-08-11T15:56:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002231434, 2023-08-11T15:56:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:56:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 56, 2023-08-11T15:56:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.009242255, 2023-08-11T15:56:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000165040 2023-08-11T15:56:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-08-11T15:56:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 86192 2023-08-11T15:56:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:56:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 74, 2023-08-11T15:56:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 73, 2023-08-11T15:56:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 158080, 2023-08-11T15:56:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 42656, 2023-08-11T15:56:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-08-11T15:56:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-08-11T15:56:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.014071867, 2023-08-11T15:56:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.005726385, 2023-08-11T15:56:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.006472609, 2023-08-11T15:56:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.001656891, 2023-08-11T15:56:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 72, 2023-08-11T15:56:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.005878451, 2023-08-11T15:56:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000081645 2023-08-11T15:56:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 158080, 2023-08-11T15:56:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 42656 2023-08-11T15:56:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-08-11T15:56:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-08-11T15:56:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-08-11T15:56:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-08-11T15:56:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:56:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-08-11T15:56:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:56:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:56:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:56:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:56:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:56:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:56:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:56:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:56:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:56:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:56:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:56:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:56:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:56:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-08-11T15:56:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:56:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-08-11T15:56:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:56:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:56:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:56:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:56:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:56:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:56:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:56:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-08-11T15:56:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:56:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:56:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:56:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-08-11T15:56:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-08-11T15:56:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-08-11T15:56:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:56:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:56:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:56:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:56:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 148823, 2023-08-11T15:56:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 27, 2023-08-11T15:56:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:56:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-08-11T15:56:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-08-11T15:56:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-08-11T15:56:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:56:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-08-11T15:56:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:56:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-08-11T15:56:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-08-11T15:56:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-08-11T15:56:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:56:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:56:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-08-11T15:56:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-08-11T15:56:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-08-11T15:56:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-08-11T15:56:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-08-11T15:56:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-08-11T15:56:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-08-11T15:56:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-08-11T15:56:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:56:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-08-11T15:56:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-08-11T15:56:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-08-11T15:56:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 75, 2023-08-11T15:56:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 147655, 2023-08-11T15:56:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-08-11T15:56:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 75, 2023-08-11T15:56:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-08-11T15:56:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-08-11T15:56:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 75, 2023-08-11T15:56:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 147655, 2023-08-11T15:56:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-08-11T15:56:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:53.595 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:56:53.985 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-08-11T15:56:53.985 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-08-11T15:56:53.986 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-08-11T15:56:53.986 DEBUG:tasks.check_counter:Getting stats from d 2023-08-11T15:56:53.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.d perf dump 2023-08-11T15:56:54.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:54 smithi134 ceph-mon[130447]: pgmap v323: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:54.724 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:56:54.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:54 smithi032 ceph-mon[97103]: pgmap v323: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:54.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:54 smithi192 ceph-mon[130806]: pgmap v323: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:56.448 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:56:56.449 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:56:56.449 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 4, 2023-08-11T15:56:56.449 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 4, 2023-08-11T15:56:56.449 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 2944, 2023-08-11T15:56:56.449 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 448, 2023-08-11T15:56:56.450 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-08-11T15:56:56.450 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-08-11T15:56:56.450 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.004336807, 2023-08-11T15:56:56.450 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000078821, 2023-08-11T15:56:56.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001353444, 2023-08-11T15:56:56.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:56:56.451 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:56.451 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 3, 2023-08-11T15:56:56.451 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.001369491, 2023-08-11T15:56:56.452 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000456497 2023-08-11T15:56:56.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.452 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:56.452 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:56.452 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:56.453 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:56.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.453 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-11T15:56:56.453 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T15:56:56.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.454 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:56:56.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 221, 2023-08-11T15:56:56.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 210, 2023-08-11T15:56:56.454 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 1545600, 2023-08-11T15:56:56.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 126032, 2023-08-11T15:56:56.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-08-11T15:56:56.455 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-08-11T15:56:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.048790708, 2023-08-11T15:56:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.016753505, 2023-08-11T15:56:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.037791302, 2023-08-11T15:56:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.004550621, 2023-08-11T15:56:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 209, 2023-08-11T15:56:56.458 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.021589181, 2023-08-11T15:56:56.458 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000103297 2023-08-11T15:56:56.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.458 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:56.458 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:56.459 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:56.459 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:56.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.459 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1545600, 2023-08-11T15:56:56.459 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 126032 2023-08-11T15:56:56.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.460 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:56:56.460 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2023-08-11T15:56:56.460 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 160, 2023-08-11T15:56:56.460 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:56:56.460 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 180464, 2023-08-11T15:56:56.461 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2023-08-11T15:56:56.461 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 3, 2023-08-11T15:56:56.461 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.016577642, 2023-08-11T15:56:56.461 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.010654328, 2023-08-11T15:56:56.461 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002669941, 2023-08-11T15:56:56.462 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:56:56.462 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:56.462 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 157, 2023-08-11T15:56:56.462 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.021322011, 2023-08-11T15:56:56.462 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000135808 2023-08-11T15:56:56.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.463 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:56.463 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:56.463 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:56.463 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:56.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.464 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:56:56.464 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 180464 2023-08-11T15:56:56.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.464 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-08-11T15:56:56.465 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-08-11T15:56:56.465 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-08-11T15:56:56.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.465 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-08-11T15:56:56.465 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:56:56.466 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-08-11T15:56:56.466 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:56:56.466 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:56:56.466 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:56:56.466 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:56:56.466 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:56:56.467 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:56:56.467 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:56:56.467 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:56:56.467 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:56:56.467 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:56:56.468 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:56:56.468 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:56:56.468 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:56:56.468 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-08-11T15:56:56.468 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:56:56.469 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-08-11T15:56:56.469 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:56:56.469 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:56:56.469 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:56:56.469 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:56:56.470 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:56:56.470 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:56:56.470 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:56:56.470 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-08-11T15:56:56.470 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:56:56.470 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:56:56.471 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:56:56.471 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-08-11T15:56:56.471 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-08-11T15:56:56.471 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-08-11T15:56:56.471 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:56:56.472 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:56:56.472 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:56:56.472 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:56:56.472 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 148977, 2023-08-11T15:56:56.472 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 30, 2023-08-11T15:56:56.473 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:56:56.473 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-08-11T15:56:56.473 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-08-11T15:56:56.473 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-08-11T15:56:56.473 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:56:56.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-08-11T15:56:56.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:56:56.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-08-11T15:56:56.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-08-11T15:56:56.474 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-08-11T15:56:56.475 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:56:56.475 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:56:56.475 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-08-11T15:56:56.475 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-08-11T15:56:56.475 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-08-11T15:56:56.476 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-08-11T15:56:56.476 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-08-11T15:56:56.476 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-08-11T15:56:56.476 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-08-11T15:56:56.476 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-08-11T15:56:56.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:56.477 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:56:56.477 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-08-11T15:56:56.477 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-08-11T15:56:56.477 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-08-11T15:56:56.477 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 223, 2023-08-11T15:56:56.478 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1514455, 2023-08-11T15:56:56.478 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-08-11T15:56:56.478 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 223, 2023-08-11T15:56:56.478 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-08-11T15:56:56.478 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-08-11T15:56:56.479 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 223, 2023-08-11T15:56:56.479 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1514455, 2023-08-11T15:56:56.479 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-08-11T15:56:56.479 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:56.479 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:56.480 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:56.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:56.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:56.480 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:56:56.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:56 smithi134 ceph-mon[130447]: pgmap v324: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:56.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:56 smithi032 ceph-mon[97103]: pgmap v324: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:56.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:56 smithi192 ceph-mon[130806]: pgmap v324: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:56.920 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-08-11T15:56:56.921 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-08-11T15:56:56.921 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-08-11T15:56:56.921 DEBUG:tasks.check_counter:Getting stats from g 2023-08-11T15:56:56.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.g perf dump 2023-08-11T15:56:57.661 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:56:58.170 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:56:58 smithi032 ceph-mon[97103]: pgmap v325: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:58.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:56:58 smithi192 ceph-mon[130806]: pgmap v325: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:58.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:56:58 smithi134 ceph-mon[130447]: pgmap v325: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:56:59.118 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:56:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:56:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 4, 2023-08-11T15:56:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 4, 2023-08-11T15:56:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 2944, 2023-08-11T15:56:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 448, 2023-08-11T15:56:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-08-11T15:56:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-08-11T15:56:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.004309768, 2023-08-11T15:56:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000084393, 2023-08-11T15:56:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001127913, 2023-08-11T15:56:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:56:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 3, 2023-08-11T15:56:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.001251546, 2023-08-11T15:56:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000417182 2023-08-11T15:56:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-11T15:56:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T15:56:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:56:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 219, 2023-08-11T15:56:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 208, 2023-08-11T15:56:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 1545056, 2023-08-11T15:56:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 124784, 2023-08-11T15:56:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-08-11T15:56:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-08-11T15:56:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.049092259, 2023-08-11T15:56:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.016438029, 2023-08-11T15:56:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.039577544, 2023-08-11T15:56:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.004594358, 2023-08-11T15:56:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 207, 2023-08-11T15:56:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.016104772, 2023-08-11T15:56:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000077800 2023-08-11T15:56:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1545056, 2023-08-11T15:56:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 124784 2023-08-11T15:56:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:56:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2023-08-11T15:56:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 159, 2023-08-11T15:56:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:56:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 180288, 2023-08-11T15:56:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2023-08-11T15:56:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 3, 2023-08-11T15:56:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.016972306, 2023-08-11T15:56:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.011061185, 2023-08-11T15:56:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002700132, 2023-08-11T15:56:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:56:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:56:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 156, 2023-08-11T15:56:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.018008992, 2023-08-11T15:56:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000115442 2023-08-11T15:56:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:56:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:56:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 180288 2023-08-11T15:56:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-08-11T15:56:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-08-11T15:56:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-08-11T15:56:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-08-11T15:56:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:56:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-08-11T15:56:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:56:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:56:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:56:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:56:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:56:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:56:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:56:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:56:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:56:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:56:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:56:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:56:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:56:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-08-11T15:56:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:56:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-08-11T15:56:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:56:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:56:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:56:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:56:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:56:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:56:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:56:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-08-11T15:56:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:56:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:56:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:56:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-08-11T15:56:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-08-11T15:56:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-08-11T15:56:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:56:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:56:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:56:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:56:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 150008, 2023-08-11T15:56:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 31, 2023-08-11T15:56:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:56:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-08-11T15:56:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-08-11T15:56:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-08-11T15:56:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:56:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-08-11T15:56:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:56:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-08-11T15:56:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-08-11T15:56:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-08-11T15:56:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:56:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:56:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-08-11T15:56:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-08-11T15:56:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-08-11T15:56:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-08-11T15:56:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-08-11T15:56:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-08-11T15:56:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-08-11T15:56:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-08-11T15:56:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:56:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:56:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-08-11T15:56:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-08-11T15:56:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-08-11T15:56:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 221, 2023-08-11T15:56:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1514195, 2023-08-11T15:56:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-08-11T15:56:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 221, 2023-08-11T15:56:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-08-11T15:56:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-08-11T15:56:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 221, 2023-08-11T15:56:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1514195, 2023-08-11T15:56:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-08-11T15:56:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:56:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:56:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:56:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:56:59.149 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:56:59.556 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-08-11T15:56:59.556 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-08-11T15:56:59.557 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-08-11T15:56:59.557 DEBUG:tasks.check_counter:Getting stats from j 2023-08-11T15:56:59.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.j perf dump 2023-08-11T15:57:00.298 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T15:57:00.642 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:00 smithi032 ceph-mon[97103]: pgmap v326: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:57:00.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:00 smithi134 ceph-mon[130447]: pgmap v326: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:57:00.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:00 smithi192 ceph-mon[130806]: pgmap v326: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2023-08-11T15:57:01.653 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:01 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:57:01.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:01 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:57:01.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:01 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T15:57:01.884 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-08-11T15:57:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2023-08-11T15:57:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2023-08-11T15:57:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2023-08-11T15:57:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2023-08-11T15:57:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-08-11T15:57:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-08-11T15:57:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.003668712, 2023-08-11T15:57:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2023-08-11T15:57:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000943421, 2023-08-11T15:57:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:57:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:57:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:57:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:57:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:57:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:57:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T15:57:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T15:57:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2023-08-11T15:57:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 162, 2023-08-11T15:57:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:57:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 180816, 2023-08-11T15:57:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 3, 2023-08-11T15:57:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.016966831, 2023-08-11T15:57:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.011059376, 2023-08-11T15:57:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002945447, 2023-08-11T15:57:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 159, 2023-08-11T15:57:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.017781599, 2023-08-11T15:57:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000111833 2023-08-11T15:57:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:57:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:57:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:57:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:57:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 180816 2023-08-11T15:57:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 227, 2023-08-11T15:57:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 214, 2023-08-11T15:57:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 1549088, 2023-08-11T15:57:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 127728, 2023-08-11T15:57:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-08-11T15:57:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-08-11T15:57:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.049650561, 2023-08-11T15:57:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.016675459, 2023-08-11T15:57:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.039422314, 2023-08-11T15:57:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.004539997, 2023-08-11T15:57:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 214, 2023-08-11T15:57:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.025668291, 2023-08-11T15:57:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000119945 2023-08-11T15:57:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:57:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:57:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:57:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1549088, 2023-08-11T15:57:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 127728 2023-08-11T15:57:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-08-11T15:57:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-08-11T15:57:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-08-11T15:57:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-08-11T15:57:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-08-11T15:57:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 148994, 2023-08-11T15:57:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 31, 2023-08-11T15:57:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-08-11T15:57:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-08-11T15:57:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-08-11T15:57:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-08-11T15:57:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-08-11T15:57:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-08-11T15:57:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-08-11T15:57:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-08-11T15:57:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-08-11T15:57:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-08-11T15:57:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-08-11T15:57:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-08-11T15:57:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-08-11T15:57:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 225, 2023-08-11T15:57:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1514715, 2023-08-11T15:57:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 225, 2023-08-11T15:57:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-08-11T15:57:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-08-11T15:57:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 225, 2023-08-11T15:57:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1514715, 2023-08-11T15:57:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-08-11T15:57:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-08-11T15:57:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-08-11T15:57:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-08-11T15:57:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:57:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-08-11T15:57:01.914 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-08-11T15:57:02.298 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-08-11T15:57:02.299 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-08-11T15:57:02.299 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-08-11T15:57:02.299 DEBUG:tasks.check_counter:Getting stats from b 2023-08-11T15:57:02.300 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.b perf dump 2023-08-11T15:57:02.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:02 smithi032 ceph-mon[97103]: pgmap v327: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:57:02.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:02 smithi134 ceph-mon[130447]: pgmap v327: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:57:02.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:02 smithi192 ceph-mon[130806]: pgmap v327: 129 pgs: 129 active+clean; 57 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-08-11T15:57:02.966 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:57:04.397 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-08-11T15:57:04.397 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:04.397 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 52206, 2023-08-11T15:57:04.398 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 51610, 2023-08-11T15:57:04.398 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 15509115, 2023-08-11T15:57:04.398 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 189361060, 2023-08-11T15:57:04.398 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 13, 2023-08-11T15:57:04.398 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-08-11T15:57:04.399 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 2.943941247, 2023-08-11T15:57:04.399 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.899509698, 2023-08-11T15:57:04.399 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.654324218, 2023-08-11T15:57:04.400 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 1.236643106, 2023-08-11T15:57:04.400 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:04.401 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 51609, 2023-08-11T15:57:04.401 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 53.773523704, 2023-08-11T15:57:04.401 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001041940 2023-08-11T15:57:04.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.402 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:04.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 894, 2023-08-11T15:57:04.403 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000139013, 2023-08-11T15:57:04.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000155 2023-08-11T15:57:04.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.404 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1930864, 2023-08-11T15:57:04.405 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 235984 2023-08-11T15:57:04.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.405 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:04.405 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 57187, 2023-08-11T15:57:04.406 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 57047, 2023-08-11T15:57:04.406 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 18506843, 2023-08-11T15:57:04.406 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 262671992, 2023-08-11T15:57:04.406 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 82, 2023-08-11T15:57:04.406 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-08-11T15:57:04.407 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 3.419810914, 2023-08-11T15:57:04.407 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 1.156571184, 2023-08-11T15:57:04.407 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.693191598, 2023-08-11T15:57:04.407 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 1.339231759, 2023-08-11T15:57:04.407 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:04.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 57044, 2023-08-11T15:57:04.408 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 40.113496737, 2023-08-11T15:57:04.408 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000703202 2023-08-11T15:57:04.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.408 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:04.408 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 284, 2023-08-11T15:57:04.409 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000039537, 2023-08-11T15:57:04.409 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000139 2023-08-11T15:57:04.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.409 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 86368, 2023-08-11T15:57:04.409 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 138640 2023-08-11T15:57:04.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.410 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:04.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 299965, 2023-08-11T15:57:04.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 497668, 2023-08-11T15:57:04.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 95490453, 2023-08-11T15:57:04.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 479364087, 2023-08-11T15:57:04.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 14, 2023-08-11T15:57:04.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-08-11T15:57:04.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 16.026576678, 2023-08-11T15:57:04.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 6.997912198, 2023-08-11T15:57:04.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 10.479115484, 2023-08-11T15:57:04.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 1.040937472, 2023-08-11T15:57:04.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:04.412 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 497667, 2023-08-11T15:57:04.412 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 68.223815591, 2023-08-11T15:57:04.413 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000137087 2023-08-11T15:57:04.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.413 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:04.413 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 594302, 2023-08-11T15:57:04.413 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.078104688, 2023-08-11T15:57:04.414 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000131 2023-08-11T15:57:04.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 61600, 2023-08-11T15:57:04.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 337744 2023-08-11T15:57:04.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.415 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-08-11T15:57:04.415 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-08-11T15:57:04.415 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-08-11T15:57:04.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.415 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-08-11T15:57:04.416 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-08-11T15:57:04.416 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-08-11T15:57:04.416 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 107433, 2023-08-11T15:57:04.416 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 71.724290916, 2023-08-11T15:57:04.416 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000667618 2023-08-11T15:57:04.417 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.417 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-08-11T15:57:04.417 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-08-11T15:57:04.417 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-08-11T15:57:04.418 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 38471, 2023-08-11T15:57:04.418 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6.731545629, 2023-08-11T15:57:04.418 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000174977 2023-08-11T15:57:04.418 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.418 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-08-11T15:57:04.419 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 184222, 2023-08-11T15:57:04.419 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 184222, 2023-08-11T15:57:04.419 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-08-11T15:57:04.419 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 184222, 2023-08-11T15:57:04.419 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 273.580072385, 2023-08-11T15:57:04.420 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001485056 2023-08-11T15:57:04.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.420 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-08-11T15:57:04.420 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2023-08-11T15:57:04.420 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-08-11T15:57:04.421 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-08-11T15:57:04.421 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 1848, 2023-08-11T15:57:04.421 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 22, 2023-08-11T15:57:04.421 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 26, 2023-08-11T15:57:04.421 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 2712, 2023-08-11T15:57:04.422 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 1128, 2023-08-11T15:57:04.422 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 1584, 2023-08-11T15:57:04.422 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2023-08-11T15:57:04.422 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 1101, 2023-08-11T15:57:04.422 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2023-08-11T15:57:04.423 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 63, 2023-08-11T15:57:04.423 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 64, 2023-08-11T15:57:04.423 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2023-08-11T15:57:04.423 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 233884, 2023-08-11T15:57:04.423 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 157169, 2023-08-11T15:57:04.424 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2023-08-11T15:57:04.424 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2023-08-11T15:57:04.424 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-08-11T15:57:04.424 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-08-11T15:57:04.424 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 1185, 2023-08-11T15:57:04.425 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2023-08-11T15:57:04.425 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-08-11T15:57:04.425 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2023-08-11T15:57:04.425 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2023-08-11T15:57:04.425 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2023-08-11T15:57:04.426 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2023-08-11T15:57:04.426 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-08-11T15:57:04.426 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2023-08-11T15:57:04.426 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-08-11T15:57:04.426 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 68, 2023-08-11T15:57:04.427 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 21196225, 2023-08-11T15:57:04.427 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2023-08-11T15:57:04.427 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T15:57:04.427 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-08-11T15:57:04.427 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T15:57:04.427 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-08-11T15:57:04.428 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-08-11T15:57:04.428 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T15:57:04.428 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T15:57:04.428 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2023-08-11T15:57:04.429 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2023-08-11T15:57:04.429 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 16934, 2023-08-11T15:57:04.429 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 150567, 2023-08-11T15:57:04.429 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T15:57:04.429 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 1131, 2023-08-11T15:57:04.430 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T15:57:04.430 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 57136, 2023-08-11T15:57:04.430 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 26926, 2023-08-11T15:57:04.431 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 7205, 2023-08-11T15:57:04.431 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 118863 2023-08-11T15:57:04.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.432 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-08-11T15:57:04.432 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-08-11T15:57:04.432 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-08-11T15:57:04.433 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-08-11T15:57:04.433 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2023-08-11T15:57:04.433 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2023-08-11T15:57:04.434 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-08-11T15:57:04.434 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-08-11T15:57:04.434 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-08-11T15:57:04.435 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 32764, 2023-08-11T15:57:04.435 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 32764, 2023-08-11T15:57:04.435 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-08-11T15:57:04.436 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-08-11T15:57:04.436 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-08-11T15:57:04.436 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-08-11T15:57:04.437 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-08-11T15:57:04.437 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-08-11T15:57:04.437 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-08-11T15:57:04.438 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T15:57:04.438 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2023-08-11T15:57:04.438 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-08-11T15:57:04.439 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 48, 2023-08-11T15:57:04.439 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-08-11T15:57:04.439 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-08-11T15:57:04.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.440 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-08-11T15:57:04.440 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2023-08-11T15:57:04.440 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 145439, 2023-08-11T15:57:04.441 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 143375, 2023-08-11T15:57:04.441 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 143375, 2023-08-11T15:57:04.441 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 2064, 2023-08-11T15:57:04.442 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-08-11T15:57:04.442 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 3807, 2023-08-11T15:57:04.442 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 227, 2023-08-11T15:57:04.443 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 224, 2023-08-11T15:57:04.443 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 224, 2023-08-11T15:57:04.443 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 3, 2023-08-11T15:57:04.444 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 1, 2023-08-11T15:57:04.444 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-08-11T15:57:04.444 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 6, 2023-08-11T15:57:04.445 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 499986928, 2023-08-11T15:57:04.445 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 504227342, 2023-08-11T15:57:04.446 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2023-08-11T15:57:04.446 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-08-11T15:57:04.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 53711, 2023-08-11T15:57:04.447 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 167.299609541, 2023-08-11T15:57:04.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003114810 2023-08-11T15:57:04.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.448 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2023-08-11T15:57:04.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.449 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-08-11T15:57:04.449 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 520, 2023-08-11T15:57:04.450 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 33284, 2023-08-11T15:57:04.450 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 32764, 2023-08-11T15:57:04.451 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 52, 2023-08-11T15:57:04.451 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 234, 2023-08-11T15:57:04.451 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 182, 2023-08-11T15:57:04.451 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 2712, 2023-08-11T15:57:04.451 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 65464, 2023-08-11T15:57:04.452 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 62752, 2023-08-11T15:57:04.452 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 64, 2023-08-11T15:57:04.452 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 33245, 2023-08-11T15:57:04.452 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 33181, 2023-08-11T15:57:04.452 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 75232, 2023-08-11T15:57:04.453 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207152 2023-08-11T15:57:04.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.453 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-08-11T15:57:04.453 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 244284, 2023-08-11T15:57:04.453 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2023-08-11T15:57:04.454 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 184222, 2023-08-11T15:57:04.454 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 4487, 2023-08-11T15:57:04.454 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2023-08-11T15:57:04.454 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-08-11T15:57:04.454 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 32832, 2023-08-11T15:57:04.455 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 32.721327002, 2023-08-11T15:57:04.455 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000996629 2023-08-11T15:57:04.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.455 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-08-11T15:57:04.455 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 188, 2023-08-11T15:57:04.456 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.150191479, 2023-08-11T15:57:04.456 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000798890 2023-08-11T15:57:04.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.456 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-08-11T15:57:04.456 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.457 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.457 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.457 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-08-11T15:57:04.457 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.458 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.458 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.458 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-08-11T15:57:04.458 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 41498, 2023-08-11T15:57:04.459 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 17.889754006, 2023-08-11T15:57:04.459 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000431099 2023-08-11T15:57:04.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.459 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-08-11T15:57:04.459 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.460 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.460 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.461 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-08-11T15:57:04.461 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2023-08-11T15:57:04.461 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000286078, 2023-08-11T15:57:04.462 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000143039 2023-08-11T15:57:04.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.463 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-08-11T15:57:04.463 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.464 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.464 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.465 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-08-11T15:57:04.465 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.466 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.466 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.467 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-08-11T15:57:04.467 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.468 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.468 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.469 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-08-11T15:57:04.469 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11, 2023-08-11T15:57:04.469 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002618604, 2023-08-11T15:57:04.470 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000238054 2023-08-11T15:57:04.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.470 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-08-11T15:57:04.471 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 20, 2023-08-11T15:57:04.471 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.021016119, 2023-08-11T15:57:04.471 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001050805 2023-08-11T15:57:04.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.472 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-08-11T15:57:04.472 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.473 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.473 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.474 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-08-11T15:57:04.474 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11, 2023-08-11T15:57:04.474 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.383979123, 2023-08-11T15:57:04.475 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.034907193 2023-08-11T15:57:04.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.475 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-08-11T15:57:04.476 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.476 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.476 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.477 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-08-11T15:57:04.478 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 68976, 2023-08-11T15:57:04.478 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 168.372749079, 2023-08-11T15:57:04.479 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002441033 2023-08-11T15:57:04.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.479 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-08-11T15:57:04.480 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6853, 2023-08-11T15:57:04.480 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 8.677322986, 2023-08-11T15:57:04.481 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001266207 2023-08-11T15:57:04.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.482 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-08-11T15:57:04.482 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.483 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.483 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.484 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-08-11T15:57:04.484 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.485 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.485 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.486 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-08-11T15:57:04.486 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.487 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.487 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.487 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-08-11T15:57:04.488 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5, 2023-08-11T15:57:04.488 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.067143982, 2023-08-11T15:57:04.488 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.013428796 2023-08-11T15:57:04.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.488 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-08-11T15:57:04.489 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.489 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.489 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.489 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-08-11T15:57:04.490 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2023-08-11T15:57:04.490 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.024037582, 2023-08-11T15:57:04.490 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.008012527 2023-08-11T15:57:04.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.490 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-08-11T15:57:04.491 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.491 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.491 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.491 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-08-11T15:57:04.492 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1052, 2023-08-11T15:57:04.492 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.332340591, 2023-08-11T15:57:04.492 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000315913 2023-08-11T15:57:04.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.492 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-08-11T15:57:04.493 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.493 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.493 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.493 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-08-11T15:57:04.494 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7, 2023-08-11T15:57:04.494 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.020607812, 2023-08-11T15:57:04.494 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002943973 2023-08-11T15:57:04.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.494 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-08-11T15:57:04.495 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.495 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.495 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.495 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-08-11T15:57:04.495 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 32764, 2023-08-11T15:57:04.496 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 44.916697942, 2023-08-11T15:57:04.496 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001370916 2023-08-11T15:57:04.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.496 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-08-11T15:57:04.497 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-08-11T15:57:04.497 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-08-11T15:57:04.497 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.498 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.498 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.498 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.499 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-08-11T15:57:04.499 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2023-08-11T15:57:04.499 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 6, 2023-08-11T15:57:04.499 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 4, 2023-08-11T15:57:04.499 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2023-08-11T15:57:04.500 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-08-11T15:57:04.500 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 9227, 2023-08-11T15:57:04.500 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 4613, 2023-08-11T15:57:04.500 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 616 2023-08-11T15:57:04.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.501 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-08-11T15:57:04.501 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:04.501 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:04.501 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:04.501 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:04.502 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:04.502 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:04.502 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:04.502 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:04.502 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:04.503 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:04.503 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:04.503 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:04.503 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:04.503 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:04.504 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:04.504 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:04.504 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:04.504 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:04.504 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:04.505 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:04.505 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:04.505 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:04.505 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:04.505 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:04.505 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:04.506 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:04.506 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:04.506 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:04.506 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:04.506 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:04.507 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:04.507 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-08-11T15:57:04.507 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:04.507 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:04.507 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:04.508 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:04.508 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 328113, 2023-08-11T15:57:04.508 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 385, 2023-08-11T15:57:04.508 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 616, 2023-08-11T15:57:04.508 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 7, 2023-08-11T15:57:04.509 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-08-11T15:57:04.509 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-08-11T15:57:04.509 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:04.509 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:04.509 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:04.510 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:04.510 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9416, 2023-08-11T15:57:04.510 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 180, 2023-08-11T15:57:04.510 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:04.510 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:04.511 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:04.511 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-08-11T15:57:04.511 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 3362428, 2023-08-11T15:57:04.511 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 35345, 2023-08-11T15:57:04.511 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:04.512 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-08-11T15:57:04.512 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:04.512 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-08-11T15:57:04.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.512 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-08-11T15:57:04.513 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-08-11T15:57:04.513 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-08-11T15:57:04.513 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 154581, 2023-08-11T15:57:04.513 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 528488580, 2023-08-11T15:57:04.513 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 31, 2023-08-11T15:57:04.513 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 154581, 2023-08-11T15:57:04.514 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-08-11T15:57:04.514 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 154581, 2023-08-11T15:57:04.514 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 5963.573578076, 2023-08-11T15:57:04.514 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.038578955 2023-08-11T15:57:04.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.515 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-08-11T15:57:04.515 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-08-11T15:57:04.515 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 154581, 2023-08-11T15:57:04.515 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 205136 2023-08-11T15:57:04.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.516 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 154581, 2023-08-11T15:57:04.516 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 651, 2023-08-11T15:57:04.516 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 153930, 2023-08-11T15:57:04.516 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-08-11T15:57:04.516 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-08-11T15:57:04.517 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1614, 2023-08-11T15:57:04.517 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 30626, 2023-08-11T15:57:04.517 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 651, 2023-08-11T15:57:04.517 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 54362, 2023-08-11T15:57:04.517 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 446, 2023-08-11T15:57:04.518 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-08-11T15:57:04.518 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-08-11T15:57:04.518 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-08-11T15:57:04.518 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 22, 2023-08-11T15:57:04.518 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 66437, 2023-08-11T15:57:04.519 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-08-11T15:57:04.519 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-08-11T15:57:04.519 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-08-11T15:57:04.519 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2023-08-11T15:57:04.519 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 46114, 2023-08-11T15:57:04.519 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-08-11T15:57:04.520 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-08-11T15:57:04.520 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-08-11T15:57:04.520 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-08-11T15:57:04.520 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-08-11T15:57:04.520 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-08-11T15:57:04.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T15:57:04.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-08-11T15:57:04.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-08-11T15:57:04.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-08-11T15:57:04.521 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-08-11T15:57:04.522 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-08-11T15:57:04.522 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-08-11T15:57:04.522 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-08-11T15:57:04.522 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-08-11T15:57:04.522 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-08-11T15:57:04.523 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-08-11T15:57:04.523 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-08-11T15:57:04.523 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-08-11T15:57:04.523 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-08-11T15:57:04.523 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-08-11T15:57:04.524 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-08-11T15:57:04.524 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-08-11T15:57:04.524 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-08-11T15:57:04.524 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-08-11T15:57:04.524 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-08-11T15:57:04.524 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 107, 2023-08-11T15:57:04.525 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-08-11T15:57:04.525 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 24, 2023-08-11T15:57:04.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2023-08-11T15:57:04.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2023-08-11T15:57:04.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-08-11T15:57:04.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-08-11T15:57:04.526 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 3762, 2023-08-11T15:57:04.526 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2023-08-11T15:57:04.526 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 1102 2023-08-11T15:57:04.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.527 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-08-11T15:57:04.527 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2023-08-11T15:57:04.527 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-08-11T15:57:04.527 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-08-11T15:57:04.527 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-08-11T15:57:04.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.528 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-08-11T15:57:04.528 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2023-08-11T15:57:04.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.528 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-08-11T15:57:04.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-08-11T15:57:04.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 48, 2023-08-11T15:57:04.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-08-11T15:57:04.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 24, 2023-08-11T15:57:04.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 32764, 2023-08-11T15:57:04.530 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-08-11T15:57:04.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.530 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:04.530 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:04.530 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-08-11T15:57:04.530 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:04.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 409354, 2023-08-11T15:57:04.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 92598228, 2023-08-11T15:57:04.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:04.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 409354, 2023-08-11T15:57:04.531 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:04.532 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:04.532 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 409354, 2023-08-11T15:57:04.532 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 92598228, 2023-08-11T15:57:04.532 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:04.532 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.533 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.533 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.533 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-08-11T15:57:04.534 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:04.534 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-08-11T15:57:04.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:04.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-08-11T15:57:04.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-08-11T15:57:04.535 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:04.535 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-08-11T15:57:04.535 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 154581, 2023-08-11T15:57:04.535 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 532063520, 2023-08-11T15:57:04.535 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 154581, 2023-08-11T15:57:04.535 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 532063520, 2023-08-11T15:57:04.536 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:04.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.536 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.537 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-08-11T15:57:04.537 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:04.537 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-08-11T15:57:04.537 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:04.538 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-08-11T15:57:04.538 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-08-11T15:57:04.538 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:04.538 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-08-11T15:57:04.538 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 154581, 2023-08-11T15:57:04.539 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 154581, 2023-08-11T15:57:04.539 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 154581, 2023-08-11T15:57:04.539 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 154581, 2023-08-11T15:57:04.539 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:04.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.539 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-08-11T15:57:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:04.541 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-08-11T15:57:04.541 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:04.541 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 32764, 2023-08-11T15:57:04.541 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 3574940, 2023-08-11T15:57:04.541 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:04.542 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 32764, 2023-08-11T15:57:04.542 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:04.542 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:04.542 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 651, 2023-08-11T15:57:04.542 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 3574940, 2023-08-11T15:57:04.543 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:04.543 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.543 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.543 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.543 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:04.544 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x5621db3f80a0": { 2023-08-11T15:57:04.544 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 60, 2023-08-11T15:57:04.544 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-08-11T15:57:04.544 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:04.544 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 145439, 2023-08-11T15:57:04.545 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 500033038, 2023-08-11T15:57:04.545 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:04.545 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 145439, 2023-08-11T15:57:04.545 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:04.545 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:04.546 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 53711, 2023-08-11T15:57:04.546 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 500032978, 2023-08-11T15:57:04.546 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:04.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:04.546 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:04.547 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:04.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:04.547 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-08-11T15:57:04.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: pgmap v328: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-08-11T15:57:04.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:04 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.752 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-08-11T15:57:04.752 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2023-08-11T15:57:04.753 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2023-08-11T15:57:04.753 DEBUG:tasks.check_counter:Getting stats from e 2023-08-11T15:57:04.754 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.e perf dump 2023-08-11T15:57:04.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: pgmap v328: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-08-11T15:57:04.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:04 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: pgmap v328: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-08-11T15:57:04.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:04.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:04.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:04 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:57:05.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-08-11T15:57:05.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-11T15:57:05.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:05.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:05.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:05 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-08-11T15:57:05.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-11T15:57:05.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:05.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:05.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:05 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-08-11T15:57:05.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-11T15:57:05.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:05.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:05.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:05.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:05 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.612 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-08-11T15:57:06.612 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:06.612 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 4, 2023-08-11T15:57:06.613 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 4, 2023-08-11T15:57:06.613 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2944, 2023-08-11T15:57:06.613 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 448, 2023-08-11T15:57:06.613 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2023-08-11T15:57:06.613 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 0, 2023-08-11T15:57:06.614 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.001597442, 2023-08-11T15:57:06.614 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.000035459, 2023-08-11T15:57:06.614 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.000462680, 2023-08-11T15:57:06.614 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:06.614 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:06.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2023-08-11T15:57:06.615 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000696327, 2023-08-11T15:57:06.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000232109 2023-08-11T15:57:06.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.615 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:06.616 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:06.616 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:06.616 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:06.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.616 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-11T15:57:06.616 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T15:57:06.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.617 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:06.617 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2, 2023-08-11T15:57:06.617 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 162, 2023-08-11T15:57:06.617 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:57:06.618 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 180816, 2023-08-11T15:57:06.618 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:06.618 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2023-08-11T15:57:06.618 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.009351171, 2023-08-11T15:57:06.618 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006084895, 2023-08-11T15:57:06.619 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.001256628, 2023-08-11T15:57:06.619 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:06.619 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:06.619 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 159, 2023-08-11T15:57:06.619 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.015839314, 2023-08-11T15:57:06.620 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000099618 2023-08-11T15:57:06.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.620 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:06.620 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:06.620 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:06.621 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:06.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.621 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:57:06.621 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 180816 2023-08-11T15:57:06.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.621 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:06.622 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 223, 2023-08-11T15:57:06.622 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 212, 2023-08-11T15:57:06.622 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1546144, 2023-08-11T15:57:06.622 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 127264, 2023-08-11T15:57:06.622 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 2, 2023-08-11T15:57:06.623 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2023-08-11T15:57:06.623 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.038358451, 2023-08-11T15:57:06.623 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.010888811, 2023-08-11T15:57:06.623 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.031142414, 2023-08-11T15:57:06.623 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.005220064, 2023-08-11T15:57:06.624 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:06.624 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 211, 2023-08-11T15:57:06.624 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.009678671, 2023-08-11T15:57:06.624 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000045870 2023-08-11T15:57:06.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.625 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:06.625 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:06.625 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:06.625 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:06.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.625 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1546144, 2023-08-11T15:57:06.626 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 127264 2023-08-11T15:57:06.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.626 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-08-11T15:57:06.626 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 0, 2023-08-11T15:57:06.626 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-08-11T15:57:06.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.627 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-08-11T15:57:06.627 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:06.627 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:06.627 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:06.628 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:06.628 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:06.628 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:06.628 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:06.628 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:06.629 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:06.629 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:06.629 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:06.629 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:06.629 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:06.630 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:06.630 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:06.630 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:06.630 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:06.630 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:06.631 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:06.631 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:06.631 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:06.631 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:06.631 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:06.632 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:06.632 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:06.632 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:06.632 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:06.632 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:06.633 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:06.633 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:06.633 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:06.633 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-08-11T15:57:06.633 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:06.634 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:06.634 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:06.634 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:06.634 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 148955, 2023-08-11T15:57:06.634 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 30, 2023-08-11T15:57:06.634 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:06.635 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:06.635 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-08-11T15:57:06.635 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-08-11T15:57:06.635 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:06.635 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:06.636 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:06.636 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:06.636 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 0, 2023-08-11T15:57:06.636 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 0, 2023-08-11T15:57:06.636 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:06.637 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:06.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:06.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-08-11T15:57:06.637 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 928, 2023-08-11T15:57:06.637 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 2, 2023-08-11T15:57:06.638 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:06.638 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-08-11T15:57:06.638 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:06.638 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-08-11T15:57:06.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:06.639 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:06.639 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:06.639 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-08-11T15:57:06.639 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:06.639 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 225, 2023-08-11T15:57:06.639 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1514715, 2023-08-11T15:57:06.640 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:06.640 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 225, 2023-08-11T15:57:06.640 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:06.640 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:06.640 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 225, 2023-08-11T15:57:06.641 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1514715, 2023-08-11T15:57:06.641 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:06.641 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:06.641 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:06.641 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:06.642 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:06.642 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:06.642 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-08-11T15:57:06.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-08-11T15:57:06.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-08-11T15:57:06.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: pgmap v329: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-08-11T15:57:06.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: Adjusting osd_memory_target on smithi032 to 222.0M 2023-08-11T15:57:06.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: Unable to set osd_memory_target on smithi032 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T15:57:06.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:57:06.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:57:06.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:06 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:06.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-08-11T15:57:06.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-08-11T15:57:06.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: pgmap v329: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-08-11T15:57:06.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: Adjusting osd_memory_target on smithi032 to 222.0M 2023-08-11T15:57:06.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: Unable to set osd_memory_target on smithi032 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T15:57:06.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:57:06.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:57:06.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:06 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:06.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-08-11T15:57:06.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-08-11T15:57:06.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: pgmap v329: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-08-11T15:57:06.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T15:57:06.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: Adjusting osd_memory_target on smithi032 to 222.0M 2023-08-11T15:57:06.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: Unable to set osd_memory_target on smithi032 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T15:57:06.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T15:57:06.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T15:57:06.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:06 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' 2023-08-11T15:57:06.958 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-08-11T15:57:06.959 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-08-11T15:57:06.959 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-08-11T15:57:06.959 DEBUG:tasks.check_counter:Getting stats from h 2023-08-11T15:57:06.960 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.h perf dump 2023-08-11T15:57:07.533 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:57:08.425 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:08 smithi134 ceph-mon[130447]: pgmap v330: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-08-11T15:57:08.426 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:08 smithi134 ceph-mon[130447]: osdmap e108: 12 total, 12 up, 12 in 2023-08-11T15:57:08.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:08 smithi032 ceph-mon[97103]: pgmap v330: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-08-11T15:57:08.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:08 smithi032 ceph-mon[97103]: osdmap e108: 12 total, 12 up, 12 in 2023-08-11T15:57:08.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:08 smithi192 ceph-mon[130806]: pgmap v330: 129 pgs: 129 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-08-11T15:57:08.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:08 smithi192 ceph-mon[130806]: osdmap e108: 12 total, 12 up, 12 in 2023-08-11T15:57:09.216 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-08-11T15:57:09.216 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 431, 2023-08-11T15:57:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 433, 2023-08-11T15:57:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 13620389, 2023-08-11T15:57:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 134051, 2023-08-11T15:57:09.217 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2023-08-11T15:57:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2023-08-11T15:57:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.042829055, 2023-08-11T15:57:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.010994839, 2023-08-11T15:57:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.015295739, 2023-08-11T15:57:09.218 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.012529531, 2023-08-11T15:57:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 431, 2023-08-11T15:57:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.028554404, 2023-08-11T15:57:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000066251 2023-08-11T15:57:09.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.220 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T15:57:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T15:57:09.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 97, 2023-08-11T15:57:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 133, 2023-08-11T15:57:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 24988084, 2023-08-11T15:57:09.222 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 174697, 2023-08-11T15:57:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-08-11T15:57:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-08-11T15:57:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.018197263, 2023-08-11T15:57:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.002930768, 2023-08-11T15:57:09.223 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.012335618, 2023-08-11T15:57:09.224 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.002030549, 2023-08-11T15:57:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:09.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 131, 2023-08-11T15:57:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.007432890, 2023-08-11T15:57:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000056739 2023-08-11T15:57:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:09.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2023-08-11T15:57:09.228 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 144784 2023-08-11T15:57:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.229 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 156, 2023-08-11T15:57:09.230 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 153, 2023-08-11T15:57:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 16974371, 2023-08-11T15:57:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 58734, 2023-08-11T15:57:09.231 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2023-08-11T15:57:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.022648751, 2023-08-11T15:57:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.003723221, 2023-08-11T15:57:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.015092642, 2023-08-11T15:57:09.232 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.003454187, 2023-08-11T15:57:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 153, 2023-08-11T15:57:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.009098113, 2023-08-11T15:57:09.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000059464 2023-08-11T15:57:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:09.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 172032, 2023-08-11T15:57:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 30112 2023-08-11T15:57:09.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-08-11T15:57:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-08-11T15:57:09.237 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-08-11T15:57:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-08-11T15:57:09.238 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-08-11T15:57:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-08-11T15:57:09.239 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1002, 2023-08-11T15:57:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.051809637, 2023-08-11T15:57:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000051706 2023-08-11T15:57:09.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.241 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-08-11T15:57:09.242 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-08-11T15:57:09.242 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-08-11T15:57:09.242 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.243 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.243 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.244 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-08-11T15:57:09.245 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2023-08-11T15:57:09.245 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2023-08-11T15:57:09.246 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-08-11T15:57:09.246 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.246 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.247 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.248 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-08-11T15:57:09.248 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2023-08-11T15:57:09.248 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-08-11T15:57:09.249 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-08-11T15:57:09.250 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2023-08-11T15:57:09.250 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2023-08-11T15:57:09.250 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2023-08-11T15:57:09.251 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 3072, 2023-08-11T15:57:09.251 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 1380, 2023-08-11T15:57:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 1114, 2023-08-11T15:57:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 578, 2023-08-11T15:57:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 1100, 2023-08-11T15:57:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 646, 2023-08-11T15:57:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2023-08-11T15:57:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2023-08-11T15:57:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2023-08-11T15:57:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2023-08-11T15:57:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2023-08-11T15:57:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2023-08-11T15:57:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2023-08-11T15:57:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-08-11T15:57:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-08-11T15:57:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2023-08-11T15:57:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2023-08-11T15:57:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-08-11T15:57:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2023-08-11T15:57:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2023-08-11T15:57:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2023-08-11T15:57:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2023-08-11T15:57:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-08-11T15:57:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2023-08-11T15:57:09.261 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-08-11T15:57:09.261 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 68, 2023-08-11T15:57:09.261 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 21196225, 2023-08-11T15:57:09.262 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2023-08-11T15:57:09.262 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T15:57:09.263 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-08-11T15:57:09.263 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T15:57:09.263 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-08-11T15:57:09.264 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-08-11T15:57:09.264 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T15:57:09.265 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T15:57:09.265 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2023-08-11T15:57:09.265 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2023-08-11T15:57:09.266 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2023-08-11T15:57:09.266 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2023-08-11T15:57:09.267 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T15:57:09.267 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T15:57:09.267 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T15:57:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2023-08-11T15:57:09.268 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2023-08-11T15:57:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2023-08-11T15:57:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2023-08-11T15:57:09.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.270 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-08-11T15:57:09.270 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-08-11T15:57:09.271 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-08-11T15:57:09.271 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-08-11T15:57:09.271 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2023-08-11T15:57:09.272 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2023-08-11T15:57:09.272 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-08-11T15:57:09.273 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-08-11T15:57:09.273 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-08-11T15:57:09.274 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2023-08-11T15:57:09.274 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2023-08-11T15:57:09.274 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-08-11T15:57:09.275 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-08-11T15:57:09.275 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-08-11T15:57:09.276 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-08-11T15:57:09.276 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-08-11T15:57:09.276 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-08-11T15:57:09.277 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-08-11T15:57:09.277 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T15:57:09.278 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2023-08-11T15:57:09.278 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-08-11T15:57:09.278 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2023-08-11T15:57:09.279 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-08-11T15:57:09.279 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-08-11T15:57:09.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.280 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-08-11T15:57:09.280 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2023-08-11T15:57:09.281 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2023-08-11T15:57:09.281 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2023-08-11T15:57:09.282 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2023-08-11T15:57:09.282 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 13729, 2023-08-11T15:57:09.282 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-08-11T15:57:09.283 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2023-08-11T15:57:09.283 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2023-08-11T15:57:09.283 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2023-08-11T15:57:09.284 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2023-08-11T15:57:09.284 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 3, 2023-08-11T15:57:09.285 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 4, 2023-08-11T15:57:09.285 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-08-11T15:57:09.285 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2023-08-11T15:57:09.286 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 499986928, 2023-08-11T15:57:09.286 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 504227342, 2023-08-11T15:57:09.287 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 504227282, 2023-08-11T15:57:09.287 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-08-11T15:57:09.287 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.288 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.288 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.289 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 13729 2023-08-11T15:57:09.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.290 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-08-11T15:57:09.290 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 203, 2023-08-11T15:57:09.291 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 3576, 2023-08-11T15:57:09.291 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 3373, 2023-08-11T15:57:09.292 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 50, 2023-08-11T15:57:09.292 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 60, 2023-08-11T15:57:09.292 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 10, 2023-08-11T15:57:09.293 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 3072, 2023-08-11T15:57:09.294 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 3718, 2023-08-11T15:57:09.294 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 646, 2023-08-11T15:57:09.294 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2023-08-11T15:57:09.295 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2023-08-11T15:57:09.295 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2023-08-11T15:57:09.295 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 53236, 2023-08-11T15:57:09.296 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207128 2023-08-11T15:57:09.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.297 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-08-11T15:57:09.297 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2023-08-11T15:57:09.298 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2023-08-11T15:57:09.298 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2023-08-11T15:57:09.298 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2023-08-11T15:57:09.299 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2023-08-11T15:57:09.299 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-08-11T15:57:09.299 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.300 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.300 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.304 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-08-11T15:57:09.304 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.304 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.304 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.305 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-08-11T15:57:09.305 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.306 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.306 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.307 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-08-11T15:57:09.307 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.308 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.308 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.308 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-08-11T15:57:09.309 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.309 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.309 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.310 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-08-11T15:57:09.310 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.311 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.311 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.312 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-08-11T15:57:09.312 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.312 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.313 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.313 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-08-11T15:57:09.313 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.314 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.314 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.314 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-08-11T15:57:09.314 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.315 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.315 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.315 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-08-11T15:57:09.315 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.316 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.316 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.316 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-08-11T15:57:09.316 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.317 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.317 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.317 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-08-11T15:57:09.317 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.318 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.318 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.318 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.318 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-08-11T15:57:09.318 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.319 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.319 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.319 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-08-11T15:57:09.319 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.320 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.320 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.320 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-08-11T15:57:09.320 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.321 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.322 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-08-11T15:57:09.322 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.323 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.324 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-08-11T15:57:09.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.324 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.325 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-08-11T15:57:09.325 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.325 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.325 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.326 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-08-11T15:57:09.326 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.326 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.327 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.327 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-08-11T15:57:09.327 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.327 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.328 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.329 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-08-11T15:57:09.329 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.329 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.329 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.331 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-08-11T15:57:09.331 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.331 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.332 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.332 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-08-11T15:57:09.333 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.333 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.333 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.334 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-08-11T15:57:09.334 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.334 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.334 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.335 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-08-11T15:57:09.335 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.335 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.335 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.336 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-08-11T15:57:09.336 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.336 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.337 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.337 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-08-11T15:57:09.337 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.338 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.338 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.338 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-08-11T15:57:09.338 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.339 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.339 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.339 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-08-11T15:57:09.339 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.340 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.340 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.340 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-08-11T15:57:09.340 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-08-11T15:57:09.341 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-08-11T15:57:09.341 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.341 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.341 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.341 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.342 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-08-11T15:57:09.342 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2023-08-11T15:57:09.343 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2023-08-11T15:57:09.343 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2023-08-11T15:57:09.343 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2023-08-11T15:57:09.343 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-08-11T15:57:09.344 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2023-08-11T15:57:09.344 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2023-08-11T15:57:09.344 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 2560 2023-08-11T15:57:09.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.345 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-08-11T15:57:09.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:09.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:09.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:09.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:09.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:09.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:09.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:09.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:09.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:09.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:09.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:09.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:09.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:09.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:09.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:09.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:09.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:09.349 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:09.349 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:09.350 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:09.350 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:09.350 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:09.350 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:09.351 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:09.351 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:09.351 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:09.352 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:09.352 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:09.352 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:09.352 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:09.352 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:09.353 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-08-11T15:57:09.353 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:09.353 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:09.353 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:09.354 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:09.354 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 239254, 2023-08-11T15:57:09.354 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 342, 2023-08-11T15:57:09.354 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:09.354 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:09.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-08-11T15:57:09.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-08-11T15:57:09.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:09.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:09.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:09.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:09.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9416, 2023-08-11T15:57:09.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 180, 2023-08-11T15:57:09.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:09.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:09.357 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:09.357 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-08-11T15:57:09.358 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 2801385, 2023-08-11T15:57:09.358 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 18792, 2023-08-11T15:57:09.358 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:09.358 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-08-11T15:57:09.358 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:09.359 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-08-11T15:57:09.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.359 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-08-11T15:57:09.359 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-08-11T15:57:09.359 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-08-11T15:57:09.361 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 619, 2023-08-11T15:57:09.361 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 26, 2023-08-11T15:57:09.362 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 4, 2023-08-11T15:57:09.362 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 619, 2023-08-11T15:57:09.363 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-08-11T15:57:09.363 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 619, 2023-08-11T15:57:09.364 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.615986129, 2023-08-11T15:57:09.364 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000995131 2023-08-11T15:57:09.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.365 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-08-11T15:57:09.365 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-08-11T15:57:09.366 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 619, 2023-08-11T15:57:09.366 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 620 2023-08-11T15:57:09.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.367 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 619, 2023-08-11T15:57:09.367 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 619, 2023-08-11T15:57:09.367 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2023-08-11T15:57:09.367 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-08-11T15:57:09.368 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-08-11T15:57:09.368 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 372, 2023-08-11T15:57:09.368 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2023-08-11T15:57:09.368 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 244, 2023-08-11T15:57:09.369 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2023-08-11T15:57:09.369 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2023-08-11T15:57:09.369 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-08-11T15:57:09.370 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-08-11T15:57:09.370 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-08-11T15:57:09.370 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2023-08-11T15:57:09.370 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2023-08-11T15:57:09.371 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-08-11T15:57:09.371 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-08-11T15:57:09.371 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-08-11T15:57:09.371 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2, 2023-08-11T15:57:09.371 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2023-08-11T15:57:09.372 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-08-11T15:57:09.372 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-08-11T15:57:09.372 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-08-11T15:57:09.372 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-08-11T15:57:09.373 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-08-11T15:57:09.373 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-08-11T15:57:09.373 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T15:57:09.373 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-08-11T15:57:09.373 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-08-11T15:57:09.374 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-08-11T15:57:09.374 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-08-11T15:57:09.374 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-08-11T15:57:09.374 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-08-11T15:57:09.374 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-08-11T15:57:09.375 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-08-11T15:57:09.375 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-08-11T15:57:09.375 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-08-11T15:57:09.375 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-08-11T15:57:09.376 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-08-11T15:57:09.376 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-08-11T15:57:09.376 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-08-11T15:57:09.376 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-08-11T15:57:09.376 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-08-11T15:57:09.377 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-08-11T15:57:09.377 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-08-11T15:57:09.377 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-08-11T15:57:09.377 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 108, 2023-08-11T15:57:09.377 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-08-11T15:57:09.378 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 5, 2023-08-11T15:57:09.378 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 10, 2023-08-11T15:57:09.378 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 10, 2023-08-11T15:57:09.378 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-08-11T15:57:09.379 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-08-11T15:57:09.379 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2023-08-11T15:57:09.380 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2023-08-11T15:57:09.380 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2023-08-11T15:57:09.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.380 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-08-11T15:57:09.381 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2023-08-11T15:57:09.381 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-08-11T15:57:09.381 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-08-11T15:57:09.381 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-08-11T15:57:09.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.382 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-08-11T15:57:09.382 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2023-08-11T15:57:09.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.382 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-08-11T15:57:09.382 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-08-11T15:57:09.383 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2023-08-11T15:57:09.383 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-08-11T15:57:09.383 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2023-08-11T15:57:09.383 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2023-08-11T15:57:09.383 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-08-11T15:57:09.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.384 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:09.384 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:09.385 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-08-11T15:57:09.385 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:09.386 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 680, 2023-08-11T15:57:09.386 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 55515631, 2023-08-11T15:57:09.386 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:09.387 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 680, 2023-08-11T15:57:09.387 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:09.388 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:09.388 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 680, 2023-08-11T15:57:09.389 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 55515631, 2023-08-11T15:57:09.389 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:09.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.390 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.391 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.392 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-08-11T15:57:09.392 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:09.392 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-08-11T15:57:09.393 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:09.393 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-08-11T15:57:09.393 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-08-11T15:57:09.393 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:09.393 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-08-11T15:57:09.394 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 619, 2023-08-11T15:57:09.394 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 55233760, 2023-08-11T15:57:09.394 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 619, 2023-08-11T15:57:09.395 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 55233760, 2023-08-11T15:57:09.395 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:09.395 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.395 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.395 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.396 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.396 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-08-11T15:57:09.396 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:09.397 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-08-11T15:57:09.397 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:09.397 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-08-11T15:57:09.397 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-08-11T15:57:09.397 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:09.398 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-08-11T15:57:09.398 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 619, 2023-08-11T15:57:09.398 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 619, 2023-08-11T15:57:09.398 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 619, 2023-08-11T15:57:09.398 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 619, 2023-08-11T15:57:09.399 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:09.399 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.399 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.399 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.400 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-08-11T15:57:09.400 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:09.400 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-08-11T15:57:09.401 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:09.401 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-08-11T15:57:09.401 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-08-11T15:57:09.402 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:09.402 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-08-11T15:57:09.402 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:09.402 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:09.402 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-08-11T15:57:09.403 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-08-11T15:57:09.403 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:09.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.403 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:09.404 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x5617c89d6460": { 2023-08-11T15:57:09.404 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:09.405 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-08-11T15:57:09.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:09.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-08-11T15:57:09.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-08-11T15:57:09.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:09.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-08-11T15:57:09.406 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:09.406 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:09.406 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-08-11T15:57:09.406 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-08-11T15:57:09.407 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:09.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:09.407 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:09.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:09.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.408 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:09.408 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-08-11T15:57:10.221 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-08-11T15:57:10.222 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-08-11T15:57:10.222 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-08-11T15:57:10.222 DEBUG:tasks.check_counter:Getting stats from k 2023-08-11T15:57:10.223 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.k perf dump 2023-08-11T15:57:10.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:10 smithi134 ceph-mon[130447]: pgmap v332: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-08-11T15:57:10.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:10 smithi032 ceph-mon[97103]: pgmap v332: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-08-11T15:57:10.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:10 smithi192 ceph-mon[130806]: pgmap v332: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-08-11T15:57:11.262 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T15:57:12.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:12 smithi134 ceph-mon[130447]: pgmap v333: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-08-11T15:57:12.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:12 smithi032 ceph-mon[97103]: pgmap v333: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-08-11T15:57:12.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:12 smithi192 ceph-mon[130806]: pgmap v333: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 57 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-08-11T15:57:13.091 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-08-11T15:57:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 0, 2023-08-11T15:57:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2, 2023-08-11T15:57:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 0, 2023-08-11T15:57:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 0, 2023-08-11T15:57:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2023-08-11T15:57:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 0, 2023-08-11T15:57:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.001200976, 2023-08-11T15:57:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.000000000, 2023-08-11T15:57:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.000347473, 2023-08-11T15:57:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T15:57:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T15:57:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2, 2023-08-11T15:57:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 165, 2023-08-11T15:57:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:57:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 181344, 2023-08-11T15:57:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2023-08-11T15:57:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.009071085, 2023-08-11T15:57:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.005412722, 2023-08-11T15:57:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.001855543, 2023-08-11T15:57:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 162, 2023-08-11T15:57:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.018791527, 2023-08-11T15:57:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000115997 2023-08-11T15:57:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:57:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 181344 2023-08-11T15:57:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 231, 2023-08-11T15:57:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 218, 2023-08-11T15:57:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1550176, 2023-08-11T15:57:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 130208, 2023-08-11T15:57:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 2, 2023-08-11T15:57:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2023-08-11T15:57:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.041012792, 2023-08-11T15:57:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.011626176, 2023-08-11T15:57:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.032852853, 2023-08-11T15:57:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.005632728, 2023-08-11T15:57:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 218, 2023-08-11T15:57:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.013928463, 2023-08-11T15:57:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000063892 2023-08-11T15:57:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1550176, 2023-08-11T15:57:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 130208 2023-08-11T15:57:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-08-11T15:57:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 0, 2023-08-11T15:57:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-08-11T15:57:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-08-11T15:57:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-08-11T15:57:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 149057, 2023-08-11T15:57:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 32, 2023-08-11T15:57:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-08-11T15:57:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-08-11T15:57:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 0, 2023-08-11T15:57:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 0, 2023-08-11T15:57:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-08-11T15:57:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 928, 2023-08-11T15:57:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 2, 2023-08-11T15:57:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-08-11T15:57:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-08-11T15:57:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-08-11T15:57:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-08-11T15:57:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-08-11T15:57:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-08-11T15:57:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 229, 2023-08-11T15:57:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1515235, 2023-08-11T15:57:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 229, 2023-08-11T15:57:13.130 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-08-11T15:57:13.130 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-08-11T15:57:13.130 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 229, 2023-08-11T15:57:13.130 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1515235, 2023-08-11T15:57:13.131 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-08-11T15:57:13.131 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-08-11T15:57:13.131 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-08-11T15:57:13.131 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-08-11T15:57:13.131 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:13.132 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-08-11T15:57:13.132 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-08-11T15:57:13.552 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-08-11T15:57:13.552 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-08-11T15:57:13.553 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-08-11T15:57:13.553 DEBUG:tasks.check_counter:Getting stats from c 2023-08-11T15:57:13.553 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.c perf dump 2023-08-11T15:57:14.117 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:57:14.660 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:14 smithi192 ceph-mon[130806]: pgmap v334: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:14.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:14 smithi134 ceph-mon[130447]: pgmap v334: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:14.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:14 smithi032 ceph-mon[97103]: pgmap v334: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:15.430 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-08-11T15:57:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 228, 2023-08-11T15:57:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 215, 2023-08-11T15:57:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1549360, 2023-08-11T15:57:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 128336, 2023-08-11T15:57:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-08-11T15:57:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2023-08-11T15:57:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.043020652, 2023-08-11T15:57:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.012059710, 2023-08-11T15:57:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.033704332, 2023-08-11T15:57:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005695509, 2023-08-11T15:57:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 215, 2023-08-11T15:57:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.010367877, 2023-08-11T15:57:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000048222 2023-08-11T15:57:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1549360, 2023-08-11T15:57:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 128336 2023-08-11T15:57:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2, 2023-08-11T15:57:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 163, 2023-08-11T15:57:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:57:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 180992, 2023-08-11T15:57:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2023-08-11T15:57:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.009321046, 2023-08-11T15:57:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.005661502, 2023-08-11T15:57:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001517476, 2023-08-11T15:57:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 160, 2023-08-11T15:57:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.023080255, 2023-08-11T15:57:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000144251 2023-08-11T15:57:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:57:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 180992 2023-08-11T15:57:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 0, 2023-08-11T15:57:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2, 2023-08-11T15:57:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 0, 2023-08-11T15:57:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 0, 2023-08-11T15:57:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2023-08-11T15:57:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2023-08-11T15:57:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.000357703, 2023-08-11T15:57:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000000000, 2023-08-11T15:57:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000301056, 2023-08-11T15:57:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T15:57:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T15:57:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-08-11T15:57:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2023-08-11T15:57:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-08-11T15:57:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-08-11T15:57:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-08-11T15:57:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 149034, 2023-08-11T15:57:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 32, 2023-08-11T15:57:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-08-11T15:57:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-08-11T15:57:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2023-08-11T15:57:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2023-08-11T15:57:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-08-11T15:57:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2023-08-11T15:57:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2023-08-11T15:57:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-08-11T15:57:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-08-11T15:57:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-08-11T15:57:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-08-11T15:57:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-08-11T15:57:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 226, 2023-08-11T15:57:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1514845, 2023-08-11T15:57:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 226, 2023-08-11T15:57:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-08-11T15:57:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-08-11T15:57:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 226, 2023-08-11T15:57:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1514845, 2023-08-11T15:57:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-08-11T15:57:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:15.470 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-08-11T15:57:15.760 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2023-08-11T15:57:15.761 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2023-08-11T15:57:15.761 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2023-08-11T15:57:15.761 DEBUG:tasks.check_counter:Getting stats from f 2023-08-11T15:57:15.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.f perf dump 2023-08-11T15:57:16.315 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:57:16.719 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:16 smithi192 ceph-mon[130806]: pgmap v335: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:16.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:16 smithi134 ceph-mon[130447]: pgmap v335: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:16.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:16 smithi032 ceph-mon[97103]: pgmap v335: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:17.529 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-08-11T15:57:17.529 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:17.530 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 4, 2023-08-11T15:57:17.530 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 4, 2023-08-11T15:57:17.530 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 2944, 2023-08-11T15:57:17.530 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 448, 2023-08-11T15:57:17.530 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-08-11T15:57:17.531 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2023-08-11T15:57:17.531 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.001561129, 2023-08-11T15:57:17.531 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000047704, 2023-08-11T15:57:17.531 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000671560, 2023-08-11T15:57:17.532 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:17.532 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:17.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3, 2023-08-11T15:57:17.532 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000741396, 2023-08-11T15:57:17.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000247132 2023-08-11T15:57:17.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.533 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:17.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:17.533 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:17.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:17.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.534 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-11T15:57:17.534 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T15:57:17.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.534 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:17.535 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2, 2023-08-11T15:57:17.535 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 164, 2023-08-11T15:57:17.535 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:57:17.535 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 181168, 2023-08-11T15:57:17.535 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:17.535 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2023-08-11T15:57:17.536 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.008980934, 2023-08-11T15:57:17.536 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.005805365, 2023-08-11T15:57:17.536 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001317086, 2023-08-11T15:57:17.537 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:17.537 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:17.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 161, 2023-08-11T15:57:17.537 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.014949836, 2023-08-11T15:57:17.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000092856 2023-08-11T15:57:17.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.538 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:17.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:17.538 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:17.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:17.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.539 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:57:17.539 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 181168 2023-08-11T15:57:17.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.539 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:17.539 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 226, 2023-08-11T15:57:17.540 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 215, 2023-08-11T15:57:17.540 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1546960, 2023-08-11T15:57:17.540 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 129136, 2023-08-11T15:57:17.540 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2023-08-11T15:57:17.540 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2023-08-11T15:57:17.541 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.042129681, 2023-08-11T15:57:17.541 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.012111545, 2023-08-11T15:57:17.542 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.033515141, 2023-08-11T15:57:17.542 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005726844, 2023-08-11T15:57:17.542 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:17.542 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 214, 2023-08-11T15:57:17.543 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.013559271, 2023-08-11T15:57:17.543 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000063361 2023-08-11T15:57:17.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.543 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:17.543 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:17.544 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:17.544 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:17.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.545 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1546960, 2023-08-11T15:57:17.546 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 129136 2023-08-11T15:57:17.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.546 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-08-11T15:57:17.547 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2023-08-11T15:57:17.547 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-08-11T15:57:17.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.547 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-08-11T15:57:17.548 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:17.548 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:17.548 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:17.548 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:17.548 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:17.549 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:17.549 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:17.549 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:17.549 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:17.549 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:17.550 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:17.550 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:17.550 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:17.550 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:17.550 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:17.551 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:17.551 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:17.551 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:17.551 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:17.551 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:17.551 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:17.552 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:17.552 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:17.552 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:17.552 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:17.553 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:17.553 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:17.553 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:17.553 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:17.553 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:17.554 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:17.554 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-08-11T15:57:17.554 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:17.554 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:17.554 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:17.555 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:17.555 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 148977, 2023-08-11T15:57:17.555 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 30, 2023-08-11T15:57:17.555 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:17.555 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:17.556 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-08-11T15:57:17.556 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-08-11T15:57:17.556 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:17.556 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:17.556 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:17.557 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:17.557 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2023-08-11T15:57:17.557 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2023-08-11T15:57:17.557 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:17.557 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:17.558 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:17.558 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-08-11T15:57:17.558 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2023-08-11T15:57:17.558 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2023-08-11T15:57:17.558 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:17.558 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-08-11T15:57:17.559 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:17.559 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-08-11T15:57:17.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:17.559 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:17.559 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-08-11T15:57:17.560 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-08-11T15:57:17.560 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-08-11T15:57:17.560 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 228, 2023-08-11T15:57:17.560 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1515105, 2023-08-11T15:57:17.560 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:17.561 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 228, 2023-08-11T15:57:17.561 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-08-11T15:57:17.561 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-08-11T15:57:17.561 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 228, 2023-08-11T15:57:17.561 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1515105, 2023-08-11T15:57:17.562 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-08-11T15:57:17.562 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:17.562 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:17.562 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:17.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:17.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:17.563 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-08-11T15:57:17.873 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2023-08-11T15:57:17.873 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2023-08-11T15:57:17.874 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2023-08-11T15:57:17.874 DEBUG:tasks.check_counter:Getting stats from i 2023-08-11T15:57:17.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.i perf dump 2023-08-11T15:57:18.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:18 smithi192 ceph-mon[130806]: pgmap v336: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:18.425 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:57:18.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:18 smithi134 ceph-mon[130447]: pgmap v336: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:18.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:18 smithi032 ceph-mon[97103]: pgmap v336: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-08-11T15:57:19.658 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-08-11T15:57:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 4, 2023-08-11T15:57:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 4, 2023-08-11T15:57:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 2944, 2023-08-11T15:57:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 448, 2023-08-11T15:57:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-08-11T15:57:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2023-08-11T15:57:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.001733297, 2023-08-11T15:57:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000036807, 2023-08-11T15:57:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000690283, 2023-08-11T15:57:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3, 2023-08-11T15:57:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000948117, 2023-08-11T15:57:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000316039 2023-08-11T15:57:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-08-11T15:57:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T15:57:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 224, 2023-08-11T15:57:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 213, 2023-08-11T15:57:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1546416, 2023-08-11T15:57:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 127888, 2023-08-11T15:57:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2023-08-11T15:57:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2023-08-11T15:57:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.041380008, 2023-08-11T15:57:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.012449733, 2023-08-11T15:57:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.032267318, 2023-08-11T15:57:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005283409, 2023-08-11T15:57:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 212, 2023-08-11T15:57:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.009986709, 2023-08-11T15:57:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000047107 2023-08-11T15:57:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1546416, 2023-08-11T15:57:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 127888 2023-08-11T15:57:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2, 2023-08-11T15:57:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 163, 2023-08-11T15:57:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:57:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 180992, 2023-08-11T15:57:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2023-08-11T15:57:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.008635433, 2023-08-11T15:57:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.005411021, 2023-08-11T15:57:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001378722, 2023-08-11T15:57:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 160, 2023-08-11T15:57:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.016068451, 2023-08-11T15:57:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000100427 2023-08-11T15:57:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:57:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 180992 2023-08-11T15:57:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-08-11T15:57:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2023-08-11T15:57:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-08-11T15:57:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-08-11T15:57:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-08-11T15:57:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:19.680 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:19.680 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:19.680 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:19.680 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 148977, 2023-08-11T15:57:19.680 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 30, 2023-08-11T15:57:19.681 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:19.681 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:19.681 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-08-11T15:57:19.681 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-08-11T15:57:19.681 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:19.682 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:19.682 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:19.682 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:19.682 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2023-08-11T15:57:19.682 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2023-08-11T15:57:19.683 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:19.683 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:19.683 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:19.683 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-08-11T15:57:19.683 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2023-08-11T15:57:19.684 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2023-08-11T15:57:19.684 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:19.684 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-08-11T15:57:19.684 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:19.684 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-08-11T15:57:19.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:19.685 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:19.685 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-08-11T15:57:19.685 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-08-11T15:57:19.685 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-08-11T15:57:19.685 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 226, 2023-08-11T15:57:19.686 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1514845, 2023-08-11T15:57:19.686 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:19.686 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 226, 2023-08-11T15:57:19.686 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-08-11T15:57:19.686 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-08-11T15:57:19.687 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 226, 2023-08-11T15:57:19.687 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1514845, 2023-08-11T15:57:19.687 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-08-11T15:57:19.687 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:19.687 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:19.688 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:19.688 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:19.688 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:19.688 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-08-11T15:57:19.972 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2023-08-11T15:57:19.972 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2023-08-11T15:57:19.973 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2023-08-11T15:57:19.973 DEBUG:tasks.check_counter:Getting stats from l 2023-08-11T15:57:19.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid cd36a838-385c-11ee-9b37-001a4aab830c -- ceph daemon mds.l perf dump 2023-08-11T15:57:20.526 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T15:57:20.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:20 smithi134 ceph-mon[130447]: pgmap v337: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 171 B/s wr, 2 op/s 2023-08-11T15:57:20.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:20 smithi032 ceph-mon[97103]: pgmap v337: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 171 B/s wr, 2 op/s 2023-08-11T15:57:20.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:20 smithi192 ceph-mon[130806]: pgmap v337: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 171 B/s wr, 2 op/s 2023-08-11T15:57:22.447 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-08-11T15:57:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T15:57:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 0, 2023-08-11T15:57:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2, 2023-08-11T15:57:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 0, 2023-08-11T15:57:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 0, 2023-08-11T15:57:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-08-11T15:57:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2023-08-11T15:57:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.001289455, 2023-08-11T15:57:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000000000, 2023-08-11T15:57:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000372646, 2023-08-11T15:57:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T15:57:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T15:57:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T15:57:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2, 2023-08-11T15:57:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 168, 2023-08-11T15:57:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 320, 2023-08-11T15:57:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 181872, 2023-08-11T15:57:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2023-08-11T15:57:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2023-08-11T15:57:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.009702075, 2023-08-11T15:57:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.006378227, 2023-08-11T15:57:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001343768, 2023-08-11T15:57:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T15:57:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 165, 2023-08-11T15:57:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.017315879, 2023-08-11T15:57:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000104944 2023-08-11T15:57:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T15:57:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 181872 2023-08-11T15:57:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T15:57:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 234, 2023-08-11T15:57:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 221, 2023-08-11T15:57:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1550992, 2023-08-11T15:57:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 132080, 2023-08-11T15:57:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2023-08-11T15:57:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2023-08-11T15:57:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.043907544, 2023-08-11T15:57:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.012486083, 2023-08-11T15:57:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.034740041, 2023-08-11T15:57:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005953706, 2023-08-11T15:57:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T15:57:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 221, 2023-08-11T15:57:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.020879595, 2023-08-11T15:57:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000094477 2023-08-11T15:57:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-08-11T15:57:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1550992, 2023-08-11T15:57:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 132080 2023-08-11T15:57:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-08-11T15:57:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2023-08-11T15:57:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-08-11T15:57:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-08-11T15:57:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-08-11T15:57:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-08-11T15:57:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T15:57:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-08-11T15:57:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T15:57:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-08-11T15:57:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T15:57:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T15:57:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T15:57:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T15:57:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T15:57:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-08-11T15:57:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T15:57:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T15:57:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-08-11T15:57:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-08-11T15:57:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-08-11T15:57:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-08-11T15:57:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T15:57:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T15:57:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T15:57:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T15:57:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T15:57:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-08-11T15:57:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-08-11T15:57:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-08-11T15:57:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T15:57:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T15:57:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-08-11T15:57:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-08-11T15:57:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-08-11T15:57:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-08-11T15:57:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T15:57:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-08-11T15:57:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T15:57:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-08-11T15:57:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 148977, 2023-08-11T15:57:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 30, 2023-08-11T15:57:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-08-11T15:57:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-08-11T15:57:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-08-11T15:57:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-08-11T15:57:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-08-11T15:57:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-08-11T15:57:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-08-11T15:57:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-08-11T15:57:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2023-08-11T15:57:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2023-08-11T15:57:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T15:57:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-08-11T15:57:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-08-11T15:57:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-08-11T15:57:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2023-08-11T15:57:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2023-08-11T15:57:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-08-11T15:57:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-08-11T15:57:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-08-11T15:57:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-08-11T15:57:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-08-11T15:57:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T15:57:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-08-11T15:57:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-08-11T15:57:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-08-11T15:57:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 232, 2023-08-11T15:57:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1515625, 2023-08-11T15:57:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-08-11T15:57:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 232, 2023-08-11T15:57:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-08-11T15:57:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-08-11T15:57:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 232, 2023-08-11T15:57:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1515625, 2023-08-11T15:57:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-08-11T15:57:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-08-11T15:57:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-08-11T15:57:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-08-11T15:57:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-08-11T15:57:22.478 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-08-11T15:57:22.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:22 smithi192 ceph-mon[130806]: pgmap v338: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-08-11T15:57:22.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:22 smithi192 ceph-mon[130806]: osdmap e109: 12 total, 12 up, 12 in 2023-08-11T15:57:22.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:22 smithi134 ceph-mon[130447]: pgmap v338: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-08-11T15:57:22.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:22 smithi134 ceph-mon[130447]: osdmap e109: 12 total, 12 up, 12 in 2023-08-11T15:57:22.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:22 smithi032 ceph-mon[97103]: pgmap v338: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-08-11T15:57:22.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:22 smithi032 ceph-mon[97103]: osdmap e109: 12 total, 12 up, 12 in 2023-08-11T15:57:22.881 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-08-11T15:57:22.882 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-08-11T15:57:22.882 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-08-11T15:57:22.883 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-08-11T15:57:22.893 INFO:tasks.kclient:Unmounting kernel clients... 2023-08-11T15:57:22.894 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:57:22.894 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:57:22.944 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-08-11T15:57:22.945 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T15:57:23.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:57:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 -- 172.21.15.32:0/3252507878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb640100890 msgr2=0x7fb640100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 --2- 172.21.15.32:0/3252507878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb640100890 0x7fb640100d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009990 tx=0x7fb62c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 -- 172.21.15.32:0/3252507878 shutdown_connections 2023-08-11T15:57:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 --2- 172.21.15.32:0/3252507878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb640101250 0x7fb64010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 --2- 172.21.15.32:0/3252507878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6401068a0 0x7fb640106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 --2- 172.21.15.32:0/3252507878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb640100890 0x7fb640100d10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 -- 172.21.15.32:0/3252507878 >> 172.21.15.32:0/3252507878 conn(0x7fb6400fc340 msgr2=0x7fb6400fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 -- 172.21.15.32:0/3252507878 shutdown_connections 2023-08-11T15:57:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.282+0000 7fb646fff700 1 -- 172.21.15.32:0/3252507878 wait complete. 2023-08-11T15:57:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.283+0000 7fb646fff700 1 Processor -- start 2023-08-11T15:57:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.284+0000 7fb646fff700 1 -- start start 2023-08-11T15:57:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.284+0000 7fb646fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb640100890 0x7fb64019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.284+0000 7fb646fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb640101250 0x7fb64019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.284+0000 7fb646fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6401068a0 0x7fb640196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.284+0000 7fb646fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640196e00 con 0x7fb6401068a0 2023-08-11T15:57:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.284+0000 7fb646fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640196f70 con 0x7fb640101250 2023-08-11T15:57:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.284+0000 7fb646fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640197130 con 0x7fb640100890 2023-08-11T15:57:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb644d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb640100890 0x7fb64019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb64559c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6401068a0 0x7fb640196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb644d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb640100890 0x7fb64019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.32:42516/0 (socket says 172.21.15.32:42516) 2023-08-11T15:57:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb644d9b700 1 -- 172.21.15.32:0/1212129740 learned_addr learned my addr 172.21.15.32:0/1212129740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb63ffff700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb640101250 0x7fb64019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb64559c700 1 -- 172.21.15.32:0/1212129740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb640100890 msgr2=0x7fb64019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb64559c700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb640100890 0x7fb64019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb64559c700 1 -- 172.21.15.32:0/1212129740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb640101250 msgr2=0x7fb64019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb64559c700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb640101250 0x7fb64019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb64559c700 1 -- 172.21.15.32:0/1212129740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0095f0 con 0x7fb6401068a0 2023-08-11T15:57:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb644d9b700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb640100890 0x7fb64019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:57:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.285+0000 7fb64559c700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6401068a0 0x7fb640196790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e970 tx=0x7fb63800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.286+0000 7fb63dffb700 1 -- 172.21.15.32:0/1212129740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638004220 con 0x7fb6401068a0 2023-08-11T15:57:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.286+0000 7fb63dffb700 1 -- 172.21.15.32:0/1212129740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb638004cf0 con 0x7fb6401068a0 2023-08-11T15:57:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.286+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb640197410 con 0x7fb6401068a0 2023-08-11T15:57:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.286+0000 7fb63dffb700 1 -- 172.21.15.32:0/1212129740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638005660 con 0x7fb6401068a0 2023-08-11T15:57:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.286+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6401a3710 con 0x7fb6401068a0 2023-08-11T15:57:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.287+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb640067710 con 0x7fb6401068a0 2023-08-11T15:57:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.289+0000 7fb63dffb700 1 -- 172.21.15.32:0/1212129740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fb638004750 con 0x7fb6401068a0 2023-08-11T15:57:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.290+0000 7fb63dffb700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb6300a0100 0x7fb6300a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.290+0000 7fb63dffb700 1 -- 172.21.15.32:0/1212129740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fb638014070 con 0x7fb6401068a0 2023-08-11T15:57:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.291+0000 7fb644d9b700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb6300a0100 0x7fb6300a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.291+0000 7fb644d9b700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb6300a0100 0x7fb6300a25c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb634009c10 tx=0x7fb63400b040 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.295+0000 7fb63dffb700 1 -- 172.21.15.32:0/1212129740 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb638094e10 con 0x7fb6401068a0 2023-08-11T15:57:23.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.510+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6400008d0 con 0x7fb6401068a0 2023-08-11T15:57:23.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.511+0000 7fb63dffb700 1 -- 172.21.15.32:0/1212129740 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+25072 (secure 0 0 0) 0x7fb638004e60 con 0x7fb6401068a0 2023-08-11T15:57:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:57:23.515 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":109,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:57:22.070894+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2323942786":"2023-08-12T15:46:41.134034+0000","172.21.15.32:6801/2482310260":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/1349164373":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/2111930192":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/2001360921":"2023-08-12T15:46:41.134034+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/2482310260":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:57:23.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.513+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb6300a0100 msgr2=0x7fb6300a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.513+0000 7fb646fff700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb6300a0100 0x7fb6300a25c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb634009c10 tx=0x7fb63400b040 comp rx=0 tx=0).stop 2023-08-11T15:57:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.513+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6401068a0 msgr2=0x7fb640196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.513+0000 7fb646fff700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6401068a0 0x7fb640196790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e970 tx=0x7fb63800ec80 comp rx=0 tx=0).stop 2023-08-11T15:57:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 shutdown_connections 2023-08-11T15:57:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fb6300a0100 0x7fb6300a25c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6401068a0 0x7fb640196790 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb640100890 0x7fb64019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 --2- 172.21.15.32:0/1212129740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb640101250 0x7fb64019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 >> 172.21.15.32:0/1212129740 conn(0x7fb6400fc340 msgr2=0x7fb6400fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 shutdown_connections 2023-08-11T15:57:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.514+0000 7fb646fff700 1 -- 172.21.15.32:0/1212129740 wait complete. 2023-08-11T15:57:23.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T15:40:08.599931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T15:44:17.700122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T15:44:19.121311+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T15:44:22.683220+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T15:57:23.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T15:57:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.742+0000 7f20fdfff700 1 -- 172.21.15.32:0/648952559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 msgr2=0x7f20f8100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.742+0000 7f20fdfff700 1 --2- 172.21.15.32:0/648952559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f8100cf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009990 tx=0x7f20ec009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.742+0000 7f20fdfff700 1 -- 172.21.15.32:0/648952559 shutdown_connections 2023-08-11T15:57:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.742+0000 7f20fdfff700 1 --2- 172.21.15.32:0/648952559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f8101230 0x7f20f810d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.742+0000 7f20fdfff700 1 --2- 172.21.15.32:0/648952559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8106880 0x7f20f8106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.742+0000 7f20fdfff700 1 --2- 172.21.15.32:0/648952559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f8100cf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.742+0000 7f20fdfff700 1 -- 172.21.15.32:0/648952559 >> 172.21.15.32:0/648952559 conn(0x7f20f80fc3a0 msgr2=0x7f20f80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.743+0000 7f20fdfff700 1 -- 172.21.15.32:0/648952559 shutdown_connections 2023-08-11T15:57:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.743+0000 7f20fdfff700 1 -- 172.21.15.32:0/648952559 wait complete. 2023-08-11T15:57:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 Processor -- start 2023-08-11T15:57:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 -- start start 2023-08-11T15:57:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f8101230 0x7f20f819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8106880 0x7f20f81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8199190 con 0x7f20f8100870 2023-08-11T15:57:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8199300 con 0x7f20f8106880 2023-08-11T15:57:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20fdfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81994c0 con 0x7f20f8101230 2023-08-11T15:57:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35316/0 (socket says 172.21.15.32:35316) 2023-08-11T15:57:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.744+0000 7f20f77fe700 1 -- 172.21.15.32:0/3456626457 learned_addr learned my addr 172.21.15.32:0/3456626457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f6ffd700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f8101230 0x7f20f819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f7fff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8106880 0x7f20f81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f77fe700 1 -- 172.21.15.32:0/3456626457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f8101230 msgr2=0x7f20f819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f77fe700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f8101230 0x7f20f819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f77fe700 1 -- 172.21.15.32:0/3456626457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8106880 msgr2=0x7f20f81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f77fe700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8106880 0x7f20f81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f77fe700 1 -- 172.21.15.32:0/3456626457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0095f0 con 0x7f20f8100870 2023-08-11T15:57:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f7fff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8106880 0x7f20f81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:57:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f77fe700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f819e9e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f20e800e9a0 tx=0x7f20e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f4ff9700 1 -- 172.21.15.32:0/3456626457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8009790 con 0x7f20f8100870 2023-08-11T15:57:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f4ff9700 1 -- 172.21.15.32:0/3456626457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e800cbd0 con 0x7f20f8100870 2023-08-11T15:57:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f81997a0 con 0x7f20f8100870 2023-08-11T15:57:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.745+0000 7f20f4ff9700 1 -- 172.21.15.32:0/3456626457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8018680 con 0x7f20f8100870 2023-08-11T15:57:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.746+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f81a5960 con 0x7f20f8100870 2023-08-11T15:57:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.746+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f80676a0 con 0x7f20f8100870 2023-08-11T15:57:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.748+0000 7f20f4ff9700 1 -- 172.21.15.32:0/3456626457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f20e800cd40 con 0x7f20f8100870 2023-08-11T15:57:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.750+0000 7f20f4ff9700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20e00a00c0 0x7f20e00a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.750+0000 7f20f4ff9700 1 -- 172.21.15.32:0/3456626457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f20e8014070 con 0x7f20f8100870 2023-08-11T15:57:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.750+0000 7f20f6ffd700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20e00a00c0 0x7f20e00a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.751+0000 7f20f6ffd700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20e00a00c0 0x7f20e00a2580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f20f819a1f0 tx=0x7f20ec005fd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:23.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.754+0000 7f20f4ff9700 1 -- 172.21.15.32:0/3456626457 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f20e8051c30 con 0x7f20f8100870 2023-08-11T15:57:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.971+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20f8110990 con 0x7f20f8100870 2023-08-11T15:57:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.971+0000 7f20f4ff9700 1 -- 172.21.15.32:0/3456626457 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20e80952f0 con 0x7f20f8100870 2023-08-11T15:57:23.974 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-08-11T15:57:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20e00a00c0 msgr2=0x7f20e00a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20e00a00c0 0x7f20e00a2580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f20f819a1f0 tx=0x7f20ec005fd0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 msgr2=0x7f20f819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f819e9e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f20e800e9a0 tx=0x7f20e800ecb0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 shutdown_connections 2023-08-11T15:57:23.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f20e00a00c0 0x7f20e00a2580 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f8100870 0x7f20f819e9e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20f8106880 0x7f20f81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 --2- 172.21.15.32:0/3456626457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20f8101230 0x7f20f819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 >> 172.21.15.32:0/3456626457 conn(0x7f20f80fc3a0 msgr2=0x7f20f810bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 shutdown_connections 2023-08-11T15:57:23.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:23.974+0000 7f20fdfff700 1 -- 172.21.15.32:0/3456626457 wait complete. 2023-08-11T15:57:23.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T15:57:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 -- 172.21.15.32:0/2226575549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c100890 msgr2=0x7f379c100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 --2- 172.21.15.32:0/2226575549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c100890 0x7f379c100d10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3788009990 tx=0x7f3788009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 -- 172.21.15.32:0/2226575549 shutdown_connections 2023-08-11T15:57:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 --2- 172.21.15.32:0/2226575549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c101250 0x7f379c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 --2- 172.21.15.32:0/2226575549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f379c1068a0 0x7f379c106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 --2- 172.21.15.32:0/2226575549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c100890 0x7f379c100d10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 -- 172.21.15.32:0/2226575549 >> 172.21.15.32:0/2226575549 conn(0x7f379c0fc380 msgr2=0x7f379c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 -- 172.21.15.32:0/2226575549 shutdown_connections 2023-08-11T15:57:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.199+0000 7f37a31ff700 1 -- 172.21.15.32:0/2226575549 wait complete. 2023-08-11T15:57:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.200+0000 7f37a31ff700 1 Processor -- start 2023-08-11T15:57:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.200+0000 7f37a31ff700 1 -- start start 2023-08-11T15:57:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a31ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c100890 0x7f379c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a31ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f379c101250 0x7f379c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a31ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c1068a0 0x7f379c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a31ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c1991a0 con 0x7f379c1068a0 2023-08-11T15:57:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a31ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c199310 con 0x7f379c101250 2023-08-11T15:57:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a31ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c1994d0 con 0x7f379c100890 2023-08-11T15:57:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c1068a0 0x7f379c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a0f9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c100890 0x7f379c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c1068a0 0x7f379c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35332/0 (socket says 172.21.15.32:35332) 2023-08-11T15:57:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 -- 172.21.15.32:0/1142272989 learned_addr learned my addr 172.21.15.32:0/1142272989 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f379bfff700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f379c101250 0x7f379c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 -- 172.21.15.32:0/1142272989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c100890 msgr2=0x7f379c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c100890 0x7f379c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 -- 172.21.15.32:0/1142272989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f379c101250 msgr2=0x7f379c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f379c101250 0x7f379c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.201+0000 7f37a179c700 1 -- 172.21.15.32:0/1142272989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880095f0 con 0x7f379c1068a0 2023-08-11T15:57:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.202+0000 7f37a179c700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c1068a0 0x7f379c1989d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f379400ef30 tx=0x7f379400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.202+0000 7f3799ffb700 1 -- 172.21.15.32:0/1142272989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794018070 con 0x7f379c1068a0 2023-08-11T15:57:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.202+0000 7f3799ffb700 1 -- 172.21.15.32:0/1142272989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379400f040 con 0x7f379c1068a0 2023-08-11T15:57:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.202+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c199750 con 0x7f379c1068a0 2023-08-11T15:57:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.202+0000 7f3799ffb700 1 -- 172.21.15.32:0/1142272989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379400cd00 con 0x7f379c1068a0 2023-08-11T15:57:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.202+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c1a5970 con 0x7f379c1068a0 2023-08-11T15:57:24.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.203+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c0676a0 con 0x7f379c1068a0 2023-08-11T15:57:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.205+0000 7f3799ffb700 1 -- 172.21.15.32:0/1142272989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f3794003a00 con 0x7f379c1068a0 2023-08-11T15:57:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.206+0000 7f3799ffb700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f378c0a0140 0x7f378c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.207+0000 7f3799ffb700 1 -- 172.21.15.32:0/1142272989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f37940cb510 con 0x7f379c1068a0 2023-08-11T15:57:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.207+0000 7f37a0f9b700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f378c0a0140 0x7f378c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.207+0000 7f37a0f9b700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f378c0a0140 0x7f378c0a2600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3790007b60 tx=0x7f37900073d0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.210+0000 7f3799ffb700 1 -- 172.21.15.32:0/1142272989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3794094c20 con 0x7f379c1068a0 2023-08-11T15:57:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.423+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f379c199f00 con 0x7f379c1068a0 2023-08-11T15:57:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.424+0000 7f3799ffb700 1 -- 172.21.15.32:0/1142272989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3794098240 con 0x7f379c1068a0 2023-08-11T15:57:24.427 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:57:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f378c0a0140 msgr2=0x7f378c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f378c0a0140 0x7f378c0a2600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3790007b60 tx=0x7f37900073d0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c1068a0 msgr2=0x7f379c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c1068a0 0x7f379c1989d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f379400ef30 tx=0x7f379400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 shutdown_connections 2023-08-11T15:57:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f378c0a0140 0x7f378c0a2600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c1068a0 0x7f379c1989d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c100890 0x7f379c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 --2- 172.21.15.32:0/1142272989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f379c101250 0x7f379c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 >> 172.21.15.32:0/1142272989 conn(0x7f379c0fc380 msgr2=0x7f379c10bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.426+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 shutdown_connections 2023-08-11T15:57:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.427+0000 7f37a31ff700 1 -- 172.21.15.32:0/1142272989 wait complete. 2023-08-11T15:57:24.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T15:57:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 -- 172.21.15.32:0/1511273018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc106880 msgr2=0x7feafc106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 --2- 172.21.15.32:0/1511273018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc106880 0x7feafc106c60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009990 tx=0x7feaf8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 -- 172.21.15.32:0/1511273018 shutdown_connections 2023-08-11T15:57:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 --2- 172.21.15.32:0/1511273018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc106880 0x7feafc106c60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 --2- 172.21.15.32:0/1511273018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feafc101230 0x7feafc10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 --2- 172.21.15.32:0/1511273018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feafc100870 0x7feafc100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 -- 172.21.15.32:0/1511273018 >> 172.21.15.32:0/1511273018 conn(0x7feafc0fc3a0 msgr2=0x7feafc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 -- 172.21.15.32:0/1511273018 shutdown_connections 2023-08-11T15:57:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.652+0000 7feb039ff700 1 -- 172.21.15.32:0/1511273018 wait complete. 2023-08-11T15:57:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.653+0000 7feb039ff700 1 Processor -- start 2023-08-11T15:57:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.653+0000 7feb039ff700 1 -- start start 2023-08-11T15:57:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.653+0000 7feb039ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feafc100870 0x7feafc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb039ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc101230 0x7feafc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb039ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feafc106880 0x7feafc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb039ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc199030 con 0x7feafc101230 2023-08-11T15:57:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb039ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc1991a0 con 0x7feafc100870 2023-08-11T15:57:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb039ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc199360 con 0x7feafc106880 2023-08-11T15:57:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc101230 0x7feafc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc101230 0x7feafc19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40800/0 (socket says 172.21.15.32:40800) 2023-08-11T15:57:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 -- 172.21.15.32:0/3057446098 learned_addr learned my addr 172.21.15.32:0/3057446098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb0179b700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feafc100870 0x7feafc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb01f9c700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feafc106880 0x7feafc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 -- 172.21.15.32:0/3057446098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feafc106880 msgr2=0x7feafc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feafc106880 0x7feafc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 -- 172.21.15.32:0/3057446098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feafc100870 msgr2=0x7feafc19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feafc100870 0x7feafc19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 -- 172.21.15.32:0/3057446098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80095f0 con 0x7feafc101230 2023-08-11T15:57:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb01f9c700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feafc106880 0x7feafc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:57:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.654+0000 7feb00f9a700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc101230 0x7feafc19ee90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feaec00e9a0 tx=0x7feaec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.655+0000 7feaf27fc700 1 -- 172.21.15.32:0/3057446098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec009790 con 0x7feafc101230 2023-08-11T15:57:24.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.655+0000 7feaf27fc700 1 -- 172.21.15.32:0/3057446098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaec00cbd0 con 0x7feafc101230 2023-08-11T15:57:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.655+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc199640 con 0x7feafc101230 2023-08-11T15:57:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.655+0000 7feaf27fc700 1 -- 172.21.15.32:0/3057446098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec018680 con 0x7feafc101230 2023-08-11T15:57:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.655+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc1a5940 con 0x7feafc101230 2023-08-11T15:57:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.656+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feafc0676a0 con 0x7feafc101230 2023-08-11T15:57:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.658+0000 7feaf27fc700 1 -- 172.21.15.32:0/3057446098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7feaec00cd40 con 0x7feafc101230 2023-08-11T15:57:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.659+0000 7feaf27fc700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7feae80a00c0 0x7feae80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.659+0000 7feaf27fc700 1 -- 172.21.15.32:0/3057446098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7feaec014070 con 0x7feafc101230 2023-08-11T15:57:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.659+0000 7feb0179b700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7feae80a00c0 0x7feae80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.660+0000 7feb0179b700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7feae80a00c0 0x7feae80a2580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b2e0 tx=0x7feaf8005fb0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.663+0000 7feaf27fc700 1 -- 172.21.15.32:0/3057446098 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7feaec051c30 con 0x7feafc101230 2023-08-11T15:57:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:24 smithi134 ceph-mon[130447]: pgmap v340: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:57:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:24 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1212129740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:57:24.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:24 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3456626457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:57:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:24 smithi032 ceph-mon[97103]: pgmap v340: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:57:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:24 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1212129740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:57:24.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:24 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3456626457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:57:24.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:24 smithi192 ceph-mon[130806]: pgmap v340: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:57:24.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:24 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1212129740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:57:24.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:24 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3456626457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T15:57:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.881+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feafc110920 con 0x7feafc101230 2023-08-11T15:57:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.881+0000 7feaf27fc700 1 -- 172.21.15.32:0/3057446098 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7feaec095350 con 0x7feafc101230 2023-08-11T15:57:24.884 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-08-11T15:57:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7feae80a00c0 msgr2=0x7feae80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7feae80a00c0 0x7feae80a2580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b2e0 tx=0x7feaf8005fb0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc101230 msgr2=0x7feafc19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc101230 0x7feafc19ee90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feaec00e9a0 tx=0x7feaec00ecb0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 shutdown_connections 2023-08-11T15:57:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7feae80a00c0 0x7feae80a2580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feafc106880 0x7feafc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feafc100870 0x7feafc19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 --2- 172.21.15.32:0/3057446098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feafc101230 0x7feafc19ee90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 >> 172.21.15.32:0/3057446098 conn(0x7feafc0fc3a0 msgr2=0x7feafc10bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 shutdown_connections 2023-08-11T15:57:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:24.884+0000 7feb039ff700 1 -- 172.21.15.32:0/3057446098 wait complete. 2023-08-11T15:57:24.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T15:57:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 -- 172.21.15.32:0/2860196131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0101230 msgr2=0x7f70e010d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 --2- 172.21.15.32:0/2860196131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0101230 0x7f70e010d740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b0a0 tx=0x7f70d800b3b0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 -- 172.21.15.32:0/2860196131 shutdown_connections 2023-08-11T15:57:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 --2- 172.21.15.32:0/2860196131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0101230 0x7f70e010d740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 --2- 172.21.15.32:0/2860196131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0106880 0x7f70e0106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 --2- 172.21.15.32:0/2860196131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e0100870 0x7f70e0100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 -- 172.21.15.32:0/2860196131 >> 172.21.15.32:0/2860196131 conn(0x7f70e00fc3a0 msgr2=0x7f70e00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 -- 172.21.15.32:0/2860196131 shutdown_connections 2023-08-11T15:57:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.110+0000 7f70e69ff700 1 -- 172.21.15.32:0/2860196131 wait complete. 2023-08-11T15:57:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.111+0000 7f70e69ff700 1 Processor -- start 2023-08-11T15:57:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.112+0000 7f70e69ff700 1 -- start start 2023-08-11T15:57:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.112+0000 7f70e69ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0100870 0x7f70e0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.112+0000 7f70e69ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e0101230 0x7f70e0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.112+0000 7f70e69ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0106880 0x7f70e01a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.112+0000 7f70e69ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0073990 con 0x7f70e0100870 2023-08-11T15:57:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.112+0000 7f70e69ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e01a5000 con 0x7f70e0101230 2023-08-11T15:57:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.112+0000 7f70e69ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e01a5140 con 0x7f70e0106880 2023-08-11T15:57:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0100870 0x7f70e0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70df7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e0101230 0x7f70e0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0100870 0x7f70e0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40818/0 (socket says 172.21.15.32:40818) 2023-08-11T15:57:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 -- 172.21.15.32:0/1227379567 learned_addr learned my addr 172.21.15.32:0/1227379567 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70e4f9c700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0106880 0x7f70e01a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 -- 172.21.15.32:0/1227379567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0106880 msgr2=0x7f70e01a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0106880 0x7f70e01a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 -- 172.21.15.32:0/1227379567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e0101230 msgr2=0x7f70e0073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e0101230 0x7f70e0073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 -- 172.21.15.32:0/1227379567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009d00 con 0x7f70e0100870 2023-08-11T15:57:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70e4f9c700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0106880 0x7f70e01a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:57:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.113+0000 7f70dffff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0100870 0x7f70e0072c20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e9a0 tx=0x7f70d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.114+0000 7f70dd7fa700 1 -- 172.21.15.32:0/1227379567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0009790 con 0x7f70e0100870 2023-08-11T15:57:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.114+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e01a53a0 con 0x7f70e0100870 2023-08-11T15:57:25.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.114+0000 7f70dd7fa700 1 -- 172.21.15.32:0/1227379567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d000cbd0 con 0x7f70e0100870 2023-08-11T15:57:25.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.114+0000 7f70dd7fa700 1 -- 172.21.15.32:0/1227379567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0018680 con 0x7f70e0100870 2023-08-11T15:57:25.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.114+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e01a58d0 con 0x7f70e0100870 2023-08-11T15:57:25.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.115+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e00676a0 con 0x7f70e0100870 2023-08-11T15:57:25.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.117+0000 7f70dd7fa700 1 -- 172.21.15.32:0/1227379567 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f70d000cd40 con 0x7f70e0100870 2023-08-11T15:57:25.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.118+0000 7f70dd7fa700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f70c80a00c0 0x7f70c80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.118+0000 7f70dd7fa700 1 -- 172.21.15.32:0/1227379567 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f70d0014070 con 0x7f70e0100870 2023-08-11T15:57:25.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.119+0000 7f70df7fe700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f70c80a00c0 0x7f70c80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.119+0000 7f70df7fe700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f70c80a00c0 0x7f70c80a2580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70e0074410 tx=0x7f70d4009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:25.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.122+0000 7f70dd7fa700 1 -- 172.21.15.32:0/1227379567 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f70d0051c30 con 0x7f70e0100870 2023-08-11T15:57:25.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.332+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f70e0110930 con 0x7f70e0100870 2023-08-11T15:57:25.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.333+0000 7f70dd7fa700 1 -- 172.21.15.32:0/1227379567 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v109) v1 ==== 103+0+11 (secure 0 0 0) 0x7f70d00952f0 con 0x7f70e0100870 2023-08-11T15:57:25.336 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-08-11T15:57:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f70c80a00c0 msgr2=0x7f70c80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f70c80a00c0 0x7f70c80a2580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70e0074410 tx=0x7f70d4009260 comp rx=0 tx=0).stop 2023-08-11T15:57:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0100870 msgr2=0x7f70e0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0100870 0x7f70e0072c20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e9a0 tx=0x7f70d000ecb0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 shutdown_connections 2023-08-11T15:57:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f70c80a00c0 0x7f70c80a2580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e0100870 0x7f70e0072c20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0106880 0x7f70e01a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 --2- 172.21.15.32:0/1227379567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e0101230 0x7f70e0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 >> 172.21.15.32:0/1227379567 conn(0x7f70e00fc3a0 msgr2=0x7f70e00fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 shutdown_connections 2023-08-11T15:57:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.335+0000 7f70e69ff700 1 -- 172.21.15.32:0/1227379567 wait complete. 2023-08-11T15:57:25.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T15:57:25.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 -- 172.21.15.32:0/1374688728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434100870 msgr2=0x7f3434100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1374688728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434100870 0x7f3434100cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3424009990 tx=0x7f3424009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 -- 172.21.15.32:0/1374688728 shutdown_connections 2023-08-11T15:57:25.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1374688728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434106880 0x7f3434106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1374688728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3434101230 0x7f343410d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1374688728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434100870 0x7f3434100cf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 -- 172.21.15.32:0/1374688728 >> 172.21.15.32:0/1374688728 conn(0x7f34340fc3a0 msgr2=0x7f34340fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:25.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 -- 172.21.15.32:0/1374688728 shutdown_connections 2023-08-11T15:57:25.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.561+0000 7f343b9ff700 1 -- 172.21.15.32:0/1374688728 wait complete. 2023-08-11T15:57:25.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.562+0000 7f343b9ff700 1 Processor -- start 2023-08-11T15:57:25.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.562+0000 7f343b9ff700 1 -- start start 2023-08-11T15:57:25.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.562+0000 7f343b9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3434100870 0x7f3434072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f343b9ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434101230 0x7f3434073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f343b9ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434106880 0x7f34341a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f343b9ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434073990 con 0x7f3434101230 2023-08-11T15:57:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f343b9ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34341a50b0 con 0x7f3434106880 2023-08-11T15:57:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f343b9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34341a51f0 con 0x7f3434100870 2023-08-11T15:57:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434101230 0x7f3434073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434101230 0x7f3434073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40842/0 (socket says 172.21.15.32:40842) 2023-08-11T15:57:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 -- 172.21.15.32:0/1465157607 learned_addr learned my addr 172.21.15.32:0/1465157607 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3439f9c700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434106880 0x7f34341a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f343979b700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3434100870 0x7f3434072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 -- 172.21.15.32:0/1465157607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3434100870 msgr2=0x7f3434072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3434100870 0x7f3434072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 -- 172.21.15.32:0/1465157607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434106880 msgr2=0x7f34341a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434106880 0x7f34341a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.563+0000 7f3438f9a700 1 -- 172.21.15.32:0/1465157607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430009520 con 0x7f3434101230 2023-08-11T15:57:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.564+0000 7f3438f9a700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434101230 0x7f3434073160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3424005910 tx=0x7f3424005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.564+0000 7f342a7fc700 1 -- 172.21.15.32:0/1465157607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342401d070 con 0x7f3434101230 2023-08-11T15:57:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.564+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240095f0 con 0x7f3434101230 2023-08-11T15:57:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.564+0000 7f342a7fc700 1 -- 172.21.15.32:0/1465157607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3424022470 con 0x7f3434101230 2023-08-11T15:57:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.564+0000 7f342a7fc700 1 -- 172.21.15.32:0/1465157607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424017650 con 0x7f3434101230 2023-08-11T15:57:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.564+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34341a5790 con 0x7f3434101230 2023-08-11T15:57:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.565+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34340676a0 con 0x7f3434101230 2023-08-11T15:57:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.567+0000 7f342a7fc700 1 -- 172.21.15.32:0/1465157607 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f342400fad0 con 0x7f3434101230 2023-08-11T15:57:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.569+0000 7f342a7fc700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f34200a0140 0x7f34200a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:25.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.569+0000 7f342a7fc700 1 -- 172.21.15.32:0/1465157607 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f34240cce50 con 0x7f3434101230 2023-08-11T15:57:25.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.569+0000 7f343979b700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f34200a0140 0x7f34200a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:25.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.569+0000 7f343979b700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f34200a0140 0x7f34200a2600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3430009d70 tx=0x7f3430009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:25.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.572+0000 7f342a7fc700 1 -- 172.21.15.32:0/1465157607 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3424096560 con 0x7f3434101230 2023-08-11T15:57:25.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:25 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1142272989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:25 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3057446098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:25 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1227379567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.798+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3434074150 con 0x7f3434101230 2023-08-11T15:57:25.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.799+0000 7f342a7fc700 1 -- 172.21.15.32:0/1465157607 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v109) v1 ==== 81+0+942 (secure 0 0 0) 0x7f3424027020 con 0x7f3434101230 2023-08-11T15:57:25.802 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2323942786 2023-08-12T15:46:41.134034+0000 2023-08-11T15:57:25.802 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2482310260 2023-08-12T15:46:41.134034+0000 2023-08-11T15:57:25.803 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1349164373 2023-08-12T15:46:41.134034+0000 2023-08-11T15:57:25.803 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2111930192 2023-08-12T15:46:41.134034+0000 2023-08-11T15:57:25.803 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/33820350 2023-08-12T15:37:53.986517+0000 2023-08-11T15:57:25.803 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/4253467068 2023-08-12T15:37:53.986517+0000 2023-08-11T15:57:25.803 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3287657132 2023-08-12T15:37:53.986517+0000 2023-08-11T15:57:25.804 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3253045891 2023-08-12T15:37:27.338674+0000 2023-08-11T15:57:25.804 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3969802586 2023-08-12T15:37:27.338674+0000 2023-08-11T15:57:25.804 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2001360921 2023-08-12T15:46:41.134034+0000 2023-08-11T15:57:25.804 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/114266517 2023-08-12T15:37:27.338674+0000 2023-08-11T15:57:25.804 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/114266517 2023-08-12T15:37:27.338674+0000 2023-08-11T15:57:25.805 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/4253467068 2023-08-12T15:37:53.986517+0000 2023-08-11T15:57:25.805 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3000621007 2023-08-12T15:37:27.338674+0000 2023-08-11T15:57:25.805 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2482310260 2023-08-12T15:46:41.134034+0000 2023-08-11T15:57:25.805 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1572440619 2023-08-12T15:37:53.986517+0000 2023-08-11T15:57:25.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:25 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1142272989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.806 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:25 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3057446098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:25 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1227379567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f34200a0140 msgr2=0x7f34200a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f34200a0140 0x7f34200a2600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3430009d70 tx=0x7f3430009260 comp rx=0 tx=0).stop 2023-08-11T15:57:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434101230 msgr2=0x7f3434073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434101230 0x7f3434073160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3424005910 tx=0x7f3424005740 comp rx=0 tx=0).stop 2023-08-11T15:57:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 shutdown_connections 2023-08-11T15:57:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f34200a0140 0x7f34200a2600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3434106880 0x7f34341a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3434100870 0x7f3434072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 --2- 172.21.15.32:0/1465157607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3434101230 0x7f3434073160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.801+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 >> 172.21.15.32:0/1465157607 conn(0x7f34340fc3a0 msgr2=0x7f34340fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.802+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 shutdown_connections 2023-08-11T15:57:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:25.802+0000 7f343b9ff700 1 -- 172.21.15.32:0/1465157607 wait complete. 2023-08-11T15:57:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:listed 16 entries 2023-08-11T15:57:25.822 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-11T15:57:25.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:57:25.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-11T15:57:25.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:25 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1142272989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:25 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3057446098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:25 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1227379567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T15:57:25.955 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T15:57:25.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:57:25.956 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-11T15:57:26.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:57:26.016 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-08-11T15:57:26.060 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-11T15:57:26.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:57:26.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-11T15:57:26.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T15:57:26.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-08-11T15:57:26.291 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-08-11T15:57:26.306 INFO:tasks.cephadm:Teardown begin 2023-08-11T15:57:26.306 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-11T15:57:26.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:57:26.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 -- 172.21.15.32:0/2332500439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 msgr2=0x7fc6f010d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:26.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/2332500439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 0x7fc6f010d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009f90 tx=0x7fc6e800b330 comp rx=0 tx=0).stop 2023-08-11T15:57:26.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 -- 172.21.15.32:0/2332500439 shutdown_connections 2023-08-11T15:57:26.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/2332500439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 0x7fc6f0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/2332500439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 0x7fc6f010d300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/2332500439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01023b0 0x7fc6f010a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 -- 172.21.15.32:0/2332500439 >> 172.21.15.32:0/2332500439 conn(0x7fc6f00fb230 msgr2=0x7fc6f00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 -- 172.21.15.32:0/2332500439 shutdown_connections 2023-08-11T15:57:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.515+0000 7fc6f6bff700 1 -- 172.21.15.32:0/2332500439 wait complete. 2023-08-11T15:57:26.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.516+0000 7fc6f6bff700 1 Processor -- start 2023-08-11T15:57:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f6bff700 1 -- start start 2023-08-11T15:57:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f6bff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 0x7fc6f019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f6bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01023b0 0x7fc6f019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f6bff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 0x7fc6f0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:26.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f6bff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0199150 con 0x7fc6f0101a90 2023-08-11T15:57:26.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f6bff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01992c0 con 0x7fc6f010ae60 2023-08-11T15:57:26.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f6bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0199480 con 0x7fc6f01023b0 2023-08-11T15:57:26.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f499b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 0x7fc6f019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:26.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f499b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 0x7fc6f019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40852/0 (socket says 172.21.15.32:40852) 2023-08-11T15:57:26.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f499b700 1 -- 172.21.15.32:0/3620246981 learned_addr learned my addr 172.21.15.32:0/3620246981 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:26.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6e7fff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01023b0 0x7fc6f019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.517+0000 7fc6f519c700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 0x7fc6f0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f499b700 1 -- 172.21.15.32:0/3620246981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01023b0 msgr2=0x7fc6f019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f499b700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01023b0 0x7fc6f019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f499b700 1 -- 172.21.15.32:0/3620246981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 msgr2=0x7fc6f0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f499b700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 0x7fc6f0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f499b700 1 -- 172.21.15.32:0/3620246981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8009c40 con 0x7fc6f0101a90 2023-08-11T15:57:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f519c700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 0x7fc6f0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:57:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f499b700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 0x7fc6f019ea70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00e9c0 tx=0x7fc6ec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6e5ffb700 1 -- 172.21.15.32:0/3620246981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec004a30 con 0x7fc6f0101a90 2023-08-11T15:57:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f0199760 con 0x7fc6f0101a90 2023-08-11T15:57:26.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6e5ffb700 1 -- 172.21.15.32:0/3620246981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6ec00cc80 con 0x7fc6f0101a90 2023-08-11T15:57:26.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6e5ffb700 1 -- 172.21.15.32:0/3620246981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0054e0 con 0x7fc6f0101a90 2023-08-11T15:57:26.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.518+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f01a5a60 con 0x7fc6f0101a90 2023-08-11T15:57:26.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.519+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f0067710 con 0x7fc6f0101a90 2023-08-11T15:57:26.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.521+0000 7fc6e5ffb700 1 -- 172.21.15.32:0/3620246981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fc6ec004b90 con 0x7fc6f0101a90 2023-08-11T15:57:26.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.523+0000 7fc6e5ffb700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc6d80a0100 0x7fc6d80a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.523+0000 7fc6e5ffb700 1 -- 172.21.15.32:0/3620246981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7fc6ec014070 con 0x7fc6f0101a90 2023-08-11T15:57:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.523+0000 7fc6e7fff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc6d80a0100 0x7fc6d80a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.523+0000 7fc6e7fff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc6d80a0100 0x7fc6d80a25c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6f010c360 tx=0x7fc6e0009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.526+0000 7fc6e5ffb700 1 -- 172.21.15.32:0/3620246981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc6ec095170 con 0x7fc6f0101a90 2023-08-11T15:57:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:26 smithi134 ceph-mon[130447]: pgmap v341: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:57:26.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:26 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1465157607' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:57:26.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.787+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f0199ff0 con 0x7fc6f0101a90 2023-08-11T15:57:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.788+0000 7fc6e5ffb700 1 -- 172.21.15.32:0/3620246981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11274 (secure 0 0 0) 0x7fc6f0199ff0 con 0x7fc6f0101a90 2023-08-11T15:57:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:57:26.792 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":15861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3095427395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3095427395},{"type":"v1","addr":"172.21.15.32:6841","nonce":3095427395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:54:23.348657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26032":{"gid":26032,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/4011211123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4011211123},{"type":"v1","addr":"172.21.15.134:6837","nonce":4011211123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:57:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.790+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc6d80a0100 msgr2=0x7fc6d80a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.790+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc6d80a0100 0x7fc6d80a25c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6f010c360 tx=0x7fc6e0009260 comp rx=0 tx=0).stop 2023-08-11T15:57:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.790+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 msgr2=0x7fc6f019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.790+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 0x7fc6f019ea70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00e9c0 tx=0x7fc6ec00ecd0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 shutdown_connections 2023-08-11T15:57:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fc6d80a0100 0x7fc6d80a25c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f0101a90 0x7fc6f019ea70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f010ae60 0x7fc6f0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 --2- 172.21.15.32:0/3620246981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01023b0 0x7fc6f019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 >> 172.21.15.32:0/3620246981 conn(0x7fc6f00fb230 msgr2=0x7fc6f00fb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 shutdown_connections 2023-08-11T15:57:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:26.791+0000 7fc6f6bff700 1 -- 172.21.15.32:0/3620246981 wait complete. 2023-08-11T15:57:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2023-08-11T15:57:26.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:26 smithi032 ceph-mon[97103]: pgmap v341: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:57:26.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:26 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1465157607' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:57:26.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T15:57:26.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:26 smithi192 ceph-mon[130806]: pgmap v341: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-08-11T15:57:26.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:26 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1465157607' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T15:57:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 -- 172.21.15.32:0/2612592042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1008a0 msgr2=0x7f0edc100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/2612592042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1008a0 0x7f0edc100d20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009990 tx=0x7f0ed0009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 -- 172.21.15.32:0/2612592042 shutdown_connections 2023-08-11T15:57:27.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/2612592042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc101260 0x7f0edc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/2612592042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0edc1068b0 0x7f0edc106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/2612592042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1008a0 0x7f0edc100d20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 -- 172.21.15.32:0/2612592042 >> 172.21.15.32:0/2612592042 conn(0x7f0edc0fc3d0 msgr2=0x7f0edc0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:27.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 -- 172.21.15.32:0/2612592042 shutdown_connections 2023-08-11T15:57:27.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.016+0000 7f0ee21ff700 1 -- 172.21.15.32:0/2612592042 wait complete. 2023-08-11T15:57:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.018+0000 7f0ee21ff700 1 Processor -- start 2023-08-11T15:57:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.018+0000 7f0ee21ff700 1 -- start start 2023-08-11T15:57:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.018+0000 7f0ee21ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0edc1008a0 0x7f0edc19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.018+0000 7f0ee21ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc101260 0x7f0edc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.018+0000 7f0ee21ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1068b0 0x7f0edc196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0ee21ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc196d70 con 0x7f0edc1068b0 2023-08-11T15:57:27.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0ee21ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc196ee0 con 0x7f0edc101260 2023-08-11T15:57:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0ee21ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1970a0 con 0x7f0edc1008a0 2023-08-11T15:57:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1068b0 0x7f0edc196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1068b0 0x7f0edc196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40868/0 (socket says 172.21.15.32:40868) 2023-08-11T15:57:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 -- 172.21.15.32:0/843475351 learned_addr learned my addr 172.21.15.32:0/843475351 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edb7fe700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0edc1008a0 0x7f0edc19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edaffd700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc101260 0x7f0edc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 -- 172.21.15.32:0/843475351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0edc1008a0 msgr2=0x7f0edc19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0edc1008a0 0x7f0edc19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 -- 172.21.15.32:0/843475351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc101260 msgr2=0x7f0edc19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc101260 0x7f0edc19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 -- 172.21.15.32:0/843475351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00095f0 con 0x7f0edc1068b0 2023-08-11T15:57:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.019+0000 7f0edbfff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1068b0 0x7f0edc196700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400ef30 tx=0x7f0ed400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.020+0000 7f0ed8ff9700 1 -- 172.21.15.32:0/843475351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4018070 con 0x7f0edc1068b0 2023-08-11T15:57:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.020+0000 7f0ed8ff9700 1 -- 172.21.15.32:0/843475351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed400f040 con 0x7f0edc1068b0 2023-08-11T15:57:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.020+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc197320 con 0x7f0edc1068b0 2023-08-11T15:57:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.020+0000 7f0ed8ff9700 1 -- 172.21.15.32:0/843475351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400cdb0 con 0x7f0edc1068b0 2023-08-11T15:57:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.020+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc1a3680 con 0x7f0edc1068b0 2023-08-11T15:57:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.021+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc067710 con 0x7f0edc1068b0 2023-08-11T15:57:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.023+0000 7f0ed8ff9700 1 -- 172.21.15.32:0/843475351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f0ed4003a00 con 0x7f0edc1068b0 2023-08-11T15:57:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.025+0000 7f0ed8ff9700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0ec40a0180 0x7f0ec40a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.025+0000 7f0ed8ff9700 1 -- 172.21.15.32:0/843475351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f0ed40cbf60 con 0x7f0edc1068b0 2023-08-11T15:57:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.025+0000 7f0edb7fe700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0ec40a0180 0x7f0ec40a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.026+0000 7f0edb7fe700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0ec40a0180 0x7f0ec40a2640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009be0 tx=0x7f0ecc009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.027+0000 7f0ed8ff9700 1 -- 172.21.15.32:0/843475351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0ed4095670 con 0x7f0edc1068b0 2023-08-11T15:57:27.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.287+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0edc197c10 con 0x7f0edc1068b0 2023-08-11T15:57:27.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.288+0000 7f0ed8ff9700 1 -- 172.21.15.32:0/843475351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11274 (secure 0 0 0) 0x7f0ed4098c90 con 0x7f0edc1068b0 2023-08-11T15:57:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:57:27.291 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3298780897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3298780897},{"type":"v1","addr":"172.21.15.32:6835","nonce":3298780897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1824615419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1824615419},{"type":"v1","addr":"172.21.15.32:6837","nonce":1824615419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/840045338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":840045338},{"type":"v1","addr":"172.21.15.32:6839","nonce":840045338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":15861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3095427395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3095427395},{"type":"v1","addr":"172.21.15.32:6841","nonce":3095427395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/408140602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":408140602},{"type":"v1","addr":"172.21.15.192:6833","nonce":408140602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3616880880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3616880880},{"type":"v1","addr":"172.21.15.134:6833","nonce":3616880880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3482775903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3482775903},{"type":"v1","addr":"172.21.15.192:6835","nonce":3482775903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1559453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1559453954},{"type":"v1","addr":"172.21.15.134:6835","nonce":1559453954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3370727785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3370727785},{"type":"v1","addr":"172.21.15.192:6837","nonce":3370727785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2697851094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2697851094},{"type":"v1","addr":"172.21.15.192:6839","nonce":2697851094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T15:44:19.707016+0000","modified":"2023-08-11T15:54:23.348657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/945169036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":945169036},{"type":"v1","addr":"172.21.15.134:6839","nonce":945169036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26032":{"gid":26032,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/4011211123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4011211123},{"type":"v1","addr":"172.21.15.134:6837","nonce":4011211123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T15:57:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.290+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0ec40a0180 msgr2=0x7f0ec40a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.290+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0ec40a0180 0x7f0ec40a2640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009be0 tx=0x7f0ecc009260 comp rx=0 tx=0).stop 2023-08-11T15:57:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.290+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1068b0 msgr2=0x7f0edc196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.290+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1068b0 0x7f0edc196700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400ef30 tx=0x7f0ed400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 shutdown_connections 2023-08-11T15:57:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0ec40a0180 0x7f0ec40a2640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0edc1068b0 0x7f0edc196700 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0edc1008a0 0x7f0edc19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 --2- 172.21.15.32:0/843475351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc101260 0x7f0edc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 >> 172.21.15.32:0/843475351 conn(0x7f0edc0fc3d0 msgr2=0x7f0edc0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 shutdown_connections 2023-08-11T15:57:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.291+0000 7f0ee21ff700 1 -- 172.21.15.32:0/843475351 wait complete. 2023-08-11T15:57:27.296 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2023-08-11T15:57:27.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T15:57:27.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 -- 172.21.15.32:0/756344309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 msgr2=0x7f0b1c106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 --2- 172.21.15.32:0/756344309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 0x7f0b1c106c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009990 tx=0x7f0b0c009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 -- 172.21.15.32:0/756344309 shutdown_connections 2023-08-11T15:57:27.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 --2- 172.21.15.32:0/756344309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1c101250 0x7f0b1c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 --2- 172.21.15.32:0/756344309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 0x7f0b1c106c80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 --2- 172.21.15.32:0/756344309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c100890 0x7f0b1c100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 -- 172.21.15.32:0/756344309 >> 172.21.15.32:0/756344309 conn(0x7f0b1c0fc340 msgr2=0x7f0b1c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:27.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 -- 172.21.15.32:0/756344309 shutdown_connections 2023-08-11T15:57:27.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.517+0000 7f0b21dff700 1 -- 172.21.15.32:0/756344309 wait complete. 2023-08-11T15:57:27.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.518+0000 7f0b21dff700 1 Processor -- start 2023-08-11T15:57:27.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.518+0000 7f0b21dff700 1 -- start start 2023-08-11T15:57:27.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b21dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1c100890 0x7f0b1c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b21dff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c101250 0x7f0b1c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b21dff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 0x7f0b1c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b21dff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c199140 con 0x7f0b1c101250 2023-08-11T15:57:27.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b21dff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c1992b0 con 0x7f0b1c1068a0 2023-08-11T15:57:27.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b21dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c199470 con 0x7f0b1c100890 2023-08-11T15:57:27.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c101250 0x7f0b1c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c101250 0x7f0b1c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40888/0 (socket says 172.21.15.32:40888) 2023-08-11T15:57:27.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 -- 172.21.15.32:0/1104446886 learned_addr learned my addr 172.21.15.32:0/1104446886 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:27.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1c100890 0x7f0b1c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1bfff700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 0x7f0b1c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 -- 172.21.15.32:0/1104446886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1c100890 msgr2=0x7f0b1c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1c100890 0x7f0b1c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 -- 172.21.15.32:0/1104446886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 msgr2=0x7f0b1c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 0x7f0b1c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.519+0000 7f0b1affd700 1 -- 172.21.15.32:0/1104446886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009520 con 0x7f0b1c101250 2023-08-11T15:57:27.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.520+0000 7f0b1affd700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c101250 0x7f0b1c19eed0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000eb30 tx=0x7f0b1000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.520+0000 7f0b18ff9700 1 -- 172.21.15.32:0/1104446886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10009880 con 0x7f0b1c101250 2023-08-11T15:57:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.520+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c0095f0 con 0x7f0b1c101250 2023-08-11T15:57:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.520+0000 7f0b18ff9700 1 -- 172.21.15.32:0/1104446886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b100044e0 con 0x7f0b1c101250 2023-08-11T15:57:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.520+0000 7f0b18ff9700 1 -- 172.21.15.32:0/1104446886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10003d80 con 0x7f0b1c101250 2023-08-11T15:57:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.520+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c1a5910 con 0x7f0b1c101250 2023-08-11T15:57:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.521+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c067710 con 0x7f0b1c101250 2023-08-11T15:57:27.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.523+0000 7f0b18ff9700 1 -- 172.21.15.32:0/1104446886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f0b10003f20 con 0x7f0b1c101250 2023-08-11T15:57:27.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.525+0000 7f0b18ff9700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0b040a0100 0x7f0b040a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.525+0000 7f0b18ff9700 1 -- 172.21.15.32:0/1104446886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f0b10014070 con 0x7f0b1c101250 2023-08-11T15:57:27.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.525+0000 7f0b1b7fe700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0b040a0100 0x7f0b040a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.526+0000 7f0b1b7fe700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0b040a0100 0x7f0b040a25c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00b2e0 tx=0x7f0b0c005fd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:27.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.529+0000 7f0b18ff9700 1 -- 172.21.15.32:0/1104446886 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0b100955e0 con 0x7f0b1c101250 2023-08-11T15:57:27.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:27 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/3620246981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:57:27.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:27 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/843475351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:57:27.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.742+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b1c199ea0 con 0x7f0b1c101250 2023-08-11T15:57:27.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.743+0000 7f0b18ff9700 1 -- 172.21.15.32:0/1104446886 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+25072 (secure 0 0 0) 0x7f0b1007f200 con 0x7f0b1c101250 2023-08-11T15:57:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-08-11T15:57:27.747 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":109,"fsid":"cd36a838-385c-11ee-9b37-001a4aab830c","created":"2023-08-11T15:36:56.367133+0000","modified":"2023-08-11T15:57:22.070894+0000","last_up_change":"2023-08-11T15:43:31.817623+0000","last_in_change":"2023-08-11T15:43:14.583624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T15:40:08.599931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T15:44:17.700122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T15:44:19.121311+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T15:44:22.683220+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"880cacc2-d87f-43c6-902e-9dfbab59b3c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6803","nonce":99848982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6805","nonce":99848982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6809","nonce":99848982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":99848982},{"type":"v1","addr":"172.21.15.32:6807","nonce":99848982}]},"public_addr":"172.21.15.32:6803/99848982","cluster_addr":"172.21.15.32:6805/99848982","heartbeat_back_addr":"172.21.15.32:6809/99848982","heartbeat_front_addr":"172.21.15.32:6807/99848982","state":["exists","up"]},{"osd":1,"uuid":"7af16e40-e09e-4ef1-ab51-bd03df34e972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6801","nonce":251950255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6803","nonce":251950255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6807","nonce":251950255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":251950255},{"type":"v1","addr":"172.21.15.134:6805","nonce":251950255}]},"public_addr":"172.21.15.134:6801/251950255","cluster_addr":"172.21.15.134:6803/251950255","heartbeat_back_addr":"172.21.15.134:6807/251950255","heartbeat_front_addr":"172.21.15.134:6805/251950255","state":["exists","up"]},{"osd":2,"uuid":"8861d523-118d-4cd5-b4fa-73fbfced3c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6801","nonce":2619008811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6803","nonce":2619008811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6807","nonce":2619008811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2619008811},{"type":"v1","addr":"172.21.15.192:6805","nonce":2619008811}]},"public_addr":"172.21.15.192:6801/2619008811","cluster_addr":"172.21.15.192:6803/2619008811","heartbeat_back_addr":"172.21.15.192:6807/2619008811","heartbeat_front_addr":"172.21.15.192:6805/2619008811","state":["exists","up"]},{"osd":3,"uuid":"d001e237-aefd-4118-b7e4-978433e52622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6811","nonce":862642324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6813","nonce":862642324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6817","nonce":862642324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":862642324},{"type":"v1","addr":"172.21.15.32:6815","nonce":862642324}]},"public_addr":"172.21.15.32:6811/862642324","cluster_addr":"172.21.15.32:6813/862642324","heartbeat_back_addr":"172.21.15.32:6817/862642324","heartbeat_front_addr":"172.21.15.32:6815/862642324","state":["exists","up"]},{"osd":4,"uuid":"6fc91951-f4cb-4edd-8713-d70f306ba0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6809","nonce":3721901848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6811","nonce":3721901848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6815","nonce":3721901848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3721901848},{"type":"v1","addr":"172.21.15.134:6813","nonce":3721901848}]},"public_addr":"172.21.15.134:6809/3721901848","cluster_addr":"172.21.15.134:6811/3721901848","heartbeat_back_addr":"172.21.15.134:6815/3721901848","heartbeat_front_addr":"172.21.15.134:6813/3721901848","state":["exists","up"]},{"osd":5,"uuid":"bfb4141a-f600-40c2-a097-ee05482183ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6809","nonce":3471337783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6811","nonce":3471337783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6815","nonce":3471337783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3471337783},{"type":"v1","addr":"172.21.15.192:6813","nonce":3471337783}]},"public_addr":"172.21.15.192:6809/3471337783","cluster_addr":"172.21.15.192:6811/3471337783","heartbeat_back_addr":"172.21.15.192:6815/3471337783","heartbeat_front_addr":"172.21.15.192:6813/3471337783","state":["exists","up"]},{"osd":6,"uuid":"8ba2f452-da4c-4d75-9452-fd60cdcc25a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6819","nonce":846261064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6821","nonce":846261064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6825","nonce":846261064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":846261064},{"type":"v1","addr":"172.21.15.32:6823","nonce":846261064}]},"public_addr":"172.21.15.32:6819/846261064","cluster_addr":"172.21.15.32:6821/846261064","heartbeat_back_addr":"172.21.15.32:6825/846261064","heartbeat_front_addr":"172.21.15.32:6823/846261064","state":["exists","up"]},{"osd":7,"uuid":"12bf8d04-54c5-4aa8-ae64-14779ba45002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6817","nonce":4228806294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6819","nonce":4228806294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6823","nonce":4228806294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4228806294},{"type":"v1","addr":"172.21.15.134:6821","nonce":4228806294}]},"public_addr":"172.21.15.134:6817/4228806294","cluster_addr":"172.21.15.134:6819/4228806294","heartbeat_back_addr":"172.21.15.134:6823/4228806294","heartbeat_front_addr":"172.21.15.134:6821/4228806294","state":["exists","up"]},{"osd":8,"uuid":"c9797fa6-65ca-40f5-aa93-b4120c4400ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6817","nonce":2583846672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6819","nonce":2583846672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6823","nonce":2583846672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2583846672},{"type":"v1","addr":"172.21.15.192:6821","nonce":2583846672}]},"public_addr":"172.21.15.192:6817/2583846672","cluster_addr":"172.21.15.192:6819/2583846672","heartbeat_back_addr":"172.21.15.192:6823/2583846672","heartbeat_front_addr":"172.21.15.192:6821/2583846672","state":["exists","up"]},{"osd":9,"uuid":"5b7e6dc0-e7d9-4003-a922-2764ad27b7c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6827","nonce":1556490683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6829","nonce":1556490683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6833","nonce":1556490683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":1556490683},{"type":"v1","addr":"172.21.15.32:6831","nonce":1556490683}]},"public_addr":"172.21.15.32:6827/1556490683","cluster_addr":"172.21.15.32:6829/1556490683","heartbeat_back_addr":"172.21.15.32:6833/1556490683","heartbeat_front_addr":"172.21.15.32:6831/1556490683","state":["exists","up"]},{"osd":10,"uuid":"06ac105d-13e0-4a1d-9c8b-f8ac17a40dd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6825","nonce":1937504104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6827","nonce":1937504104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6831","nonce":1937504104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1937504104},{"type":"v1","addr":"172.21.15.134:6829","nonce":1937504104}]},"public_addr":"172.21.15.134:6825/1937504104","cluster_addr":"172.21.15.134:6827/1937504104","heartbeat_back_addr":"172.21.15.134:6831/1937504104","heartbeat_front_addr":"172.21.15.134:6829/1937504104","state":["exists","up"]},{"osd":11,"uuid":"2799be08-a3bf-4a40-a968-0e091285d8c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6825","nonce":1267688108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6827","nonce":1267688108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6831","nonce":1267688108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1267688108},{"type":"v1","addr":"172.21.15.192:6829","nonce":1267688108}]},"public_addr":"172.21.15.192:6825/1267688108","cluster_addr":"172.21.15.192:6827/1267688108","heartbeat_back_addr":"172.21.15.192:6831/1267688108","heartbeat_front_addr":"172.21.15.192:6829/1267688108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:24.336831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:39:45.817540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:05.897376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:32.414803+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:40:54.315922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:14.701548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:41:40.588816+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:02.067927+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:22.489992+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:42:48.669868+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:09.901705+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T15:43:30.099710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2323942786":"2023-08-12T15:46:41.134034+0000","172.21.15.32:6801/2482310260":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/1349164373":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/2111930192":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/33820350":"2023-08-12T15:37:53.986517+0000","172.21.15.32:6800/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3287657132":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3253045891":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/3969802586":"2023-08-12T15:37:27.338674+0000","172.21.15.32:0/2001360921":"2023-08-12T15:46:41.134034+0000","172.21.15.32:6800/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/114266517":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6801/4253467068":"2023-08-12T15:37:53.986517+0000","172.21.15.32:0/3000621007":"2023-08-12T15:37:27.338674+0000","172.21.15.32:6800/2482310260":"2023-08-12T15:46:41.134034+0000","172.21.15.32:0/1572440619":"2023-08-12T15:37:53.986517+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T15:57:27.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0b040a0100 msgr2=0x7f0b040a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0b040a0100 0x7f0b040a25c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00b2e0 tx=0x7f0b0c005fd0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c101250 msgr2=0x7f0b1c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c101250 0x7f0b1c19eed0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000eb30 tx=0x7f0b1000ee40 comp rx=0 tx=0).stop 2023-08-11T15:57:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 shutdown_connections 2023-08-11T15:57:27.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f0b040a0100 0x7f0b040a25c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1068a0 0x7f0b1c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1c100890 0x7f0b1c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 --2- 172.21.15.32:0/1104446886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1c101250 0x7f0b1c19eed0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 >> 172.21.15.32:0/1104446886 conn(0x7f0b1c0fc340 msgr2=0x7f0b1c10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 shutdown_connections 2023-08-11T15:57:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.746+0000 7f0b21dff700 1 -- 172.21.15.32:0/1104446886 wait complete. 2023-08-11T15:57:27.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:27 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/3620246981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:57:27.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:27 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/843475351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:57:27.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-11T15:57:27.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:27 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/3620246981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:57:27.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:27 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/843475351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T15:57:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.970+0000 7f8f031ff700 1 -- 172.21.15.32:0/2746377648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc100870 msgr2=0x7f8efc100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.970+0000 7f8f031ff700 1 --2- 172.21.15.32:0/2746377648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc100870 0x7f8efc100cf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.970+0000 7f8f031ff700 1 -- 172.21.15.32:0/2746377648 shutdown_connections 2023-08-11T15:57:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.970+0000 7f8f031ff700 1 --2- 172.21.15.32:0/2746377648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8efc101230 0x7f8efc10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.970+0000 7f8f031ff700 1 --2- 172.21.15.32:0/2746377648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc106880 0x7f8efc106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.970+0000 7f8f031ff700 1 --2- 172.21.15.32:0/2746377648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc100870 0x7f8efc100cf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.970+0000 7f8f031ff700 1 -- 172.21.15.32:0/2746377648 >> 172.21.15.32:0/2746377648 conn(0x7f8efc0fc3a0 msgr2=0x7f8efc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.971+0000 7f8f031ff700 1 -- 172.21.15.32:0/2746377648 shutdown_connections 2023-08-11T15:57:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.971+0000 7f8f031ff700 1 -- 172.21.15.32:0/2746377648 wait complete. 2023-08-11T15:57:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 Processor -- start 2023-08-11T15:57:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 -- start start 2023-08-11T15:57:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8efc100870 0x7f8efc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc101230 0x7f8efc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc106880 0x7f8efc196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc196e00 con 0x7f8efc106880 2023-08-11T15:57:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc196f70 con 0x7f8efc101230 2023-08-11T15:57:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f031ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc197130 con 0x7f8efc100870 2023-08-11T15:57:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f0179c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc106880 0x7f8efc196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f0179c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc106880 0x7f8efc196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40910/0 (socket says 172.21.15.32:40910) 2023-08-11T15:57:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.972+0000 7f8f0179c700 1 -- 172.21.15.32:0/188588884 learned_addr learned my addr 172.21.15.32:0/188588884 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8efbfff700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc101230 0x7f8efc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f00f9b700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8efc100870 0x7f8efc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f0179c700 1 -- 172.21.15.32:0/188588884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8efc100870 msgr2=0x7f8efc19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f0179c700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8efc100870 0x7f8efc19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f0179c700 1 -- 172.21.15.32:0/188588884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc101230 msgr2=0x7f8efc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f0179c700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc101230 0x7f8efc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f0179c700 1 -- 172.21.15.32:0/188588884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80095f0 con 0x7f8efc106880 2023-08-11T15:57:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f0179c700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc106880 0x7f8efc196790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400ef30 tx=0x7f8ef400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8ef9ffb700 1 -- 172.21.15.32:0/188588884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4018070 con 0x7f8efc106880 2023-08-11T15:57:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc1973b0 con 0x7f8efc106880 2023-08-11T15:57:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8ef9ffb700 1 -- 172.21.15.32:0/188588884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ef400f040 con 0x7f8efc106880 2023-08-11T15:57:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.973+0000 7f8ef9ffb700 1 -- 172.21.15.32:0/188588884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400cd00 con 0x7f8efc106880 2023-08-11T15:57:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.974+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc1a3710 con 0x7f8efc106880 2023-08-11T15:57:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.975+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc0676a0 con 0x7f8efc106880 2023-08-11T15:57:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.977+0000 7f8ef9ffb700 1 -- 172.21.15.32:0/188588884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f8ef4003a00 con 0x7f8efc106880 2023-08-11T15:57:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.978+0000 7f8ef9ffb700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f8eec0a00c0 0x7f8eec0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.978+0000 7f8ef9ffb700 1 -- 172.21.15.32:0/188588884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9395+0+0 (secure 0 0 0) 0x7f8ef40cb520 con 0x7f8efc106880 2023-08-11T15:57:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.978+0000 7f8f00f9b700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f8eec0a00c0 0x7f8eec0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.979+0000 7f8f00f9b700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f8eec0a00c0 0x7f8eec0a2580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009be0 tx=0x7f8ef0009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:27.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:27.982+0000 7f8ef9ffb700 1 -- 172.21.15.32:0/188588884 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8ef4094c90 con 0x7f8efc106880 2023-08-11T15:57:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.241+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8efc1108c0 con 0x7f8efc106880 2023-08-11T15:57:28.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.247+0000 7f8ef9ffb700 1 -- 172.21.15.32:0/188588884 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8ef40982b0 con 0x7f8efc106880 2023-08-11T15:57:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.249+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f8eec0a00c0 msgr2=0x7f8eec0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.249+0000 7f8f031ff700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f8eec0a00c0 0x7f8eec0a2580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009be0 tx=0x7f8ef0009260 comp rx=0 tx=0).stop 2023-08-11T15:57:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc106880 msgr2=0x7f8efc196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc106880 0x7f8efc196790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400ef30 tx=0x7f8ef400c3a0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 shutdown_connections 2023-08-11T15:57:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f8eec0a00c0 0x7f8eec0a2580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc106880 0x7f8efc196790 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8efc100870 0x7f8efc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 --2- 172.21.15.32:0/188588884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc101230 0x7f8efc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 >> 172.21.15.32:0/188588884 conn(0x7f8efc0fc3a0 msgr2=0x7f8efc0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 shutdown_connections 2023-08-11T15:57:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.250+0000 7f8f031ff700 1 -- 172.21.15.32:0/188588884 wait complete. 2023-08-11T15:57:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-11T15:57:28.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-11T15:57:28.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.481+0000 7ff43d9ff700 1 -- 172.21.15.32:0/3233253342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438106880 msgr2=0x7ff438106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.481+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/3233253342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438106880 0x7ff438106c60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff428009990 tx=0x7ff428009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.481+0000 7ff43d9ff700 1 -- 172.21.15.32:0/3233253342 shutdown_connections 2023-08-11T15:57:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.481+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/3233253342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438106880 0x7ff438106c60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.481+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/3233253342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438101230 0x7ff43810d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.481+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/3233253342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff438100870 0x7ff438100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.481+0000 7ff43d9ff700 1 -- 172.21.15.32:0/3233253342 >> 172.21.15.32:0/3233253342 conn(0x7ff4380fc3a0 msgr2=0x7ff4380fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.482+0000 7ff43d9ff700 1 -- 172.21.15.32:0/3233253342 shutdown_connections 2023-08-11T15:57:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.482+0000 7ff43d9ff700 1 -- 172.21.15.32:0/3233253342 wait complete. 2023-08-11T15:57:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 Processor -- start 2023-08-11T15:57:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 -- start start 2023-08-11T15:57:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438100870 0x7ff43819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438101230 0x7ff43819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff438106880 0x7ff4381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff438199010 con 0x7ff438101230 2023-08-11T15:57:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff438199180 con 0x7ff438100870 2023-08-11T15:57:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff43d9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff438199340 con 0x7ff438106880 2023-08-11T15:57:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.483+0000 7ff4367fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438101230 0x7ff43819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438101230 0x7ff43819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40922/0 (socket says 172.21.15.32:40922) 2023-08-11T15:57:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 -- 172.21.15.32:0/4050150142 learned_addr learned my addr 172.21.15.32:0/4050150142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff436ffd700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438100870 0x7ff43819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4377fe700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff438106880 0x7ff4381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 -- 172.21.15.32:0/4050150142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff438106880 msgr2=0x7ff4381989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff438106880 0x7ff4381989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 -- 172.21.15.32:0/4050150142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438100870 msgr2=0x7ff43819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438100870 0x7ff43819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 -- 172.21.15.32:0/4050150142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4280095f0 con 0x7ff438101230 2023-08-11T15:57:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4377fe700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff438106880 0x7ff4381989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T15:57:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff4367fc700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438101230 0x7ff43819ee70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00d950 tx=0x7ff42c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff43c9fd700 1 -- 172.21.15.32:0/4050150142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c009940 con 0x7ff438101230 2023-08-11T15:57:28.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff438199620 con 0x7ff438101230 2023-08-11T15:57:28.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff43c9fd700 1 -- 172.21.15.32:0/4050150142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff42c010460 con 0x7ff438101230 2023-08-11T15:57:28.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.484+0000 7ff43c9fd700 1 -- 172.21.15.32:0/4050150142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00f620 con 0x7ff438101230 2023-08-11T15:57:28.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.485+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4381a5920 con 0x7ff438101230 2023-08-11T15:57:28.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.485+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4380676a0 con 0x7ff438101230 2023-08-11T15:57:28.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.487+0000 7ff43c9fd700 1 -- 172.21.15.32:0/4050150142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7ff42c0108d0 con 0x7ff438101230 2023-08-11T15:57:28.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.489+0000 7ff43c9fd700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff4200a00c0 0x7ff4200a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:28.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.489+0000 7ff43c9fd700 1 -- 172.21.15.32:0/4050150142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9393+0+0 (secure 0 0 0) 0x7ff42c0cb880 con 0x7ff438101230 2023-08-11T15:57:28.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.489+0000 7ff436ffd700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff4200a00c0 0x7ff4200a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:28.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.490+0000 7ff436ffd700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff4200a00c0 0x7ff4200a2580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff42800b2e0 tx=0x7ff428005fd0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:28.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.493+0000 7ff43c9fd700 1 -- 172.21.15.32:0/4050150142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff42c094f00 con 0x7ff438101230 2023-08-11T15:57:28.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-a[97079]: 2023-08-11T15:57:28.241+0000 7f973e74d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T15:57:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: pgmap v342: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2023-08-11T15:57:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/1104446886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:57:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/188588884' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T15:57:28.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-11T15:57:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T15:57:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: osdmap e110: 12 total, 12 up, 12 in 2023-08-11T15:57:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/188588884' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-11T15:57:28.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:28 smithi032 ceph-mon[97103]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-08-11T15:57:28.721 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: ignoring --setuser ceph since I am not root 2023-08-11T15:57:28.721 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: ignoring --setgroup ceph since I am not root 2023-08-11T15:57:28.722 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mds[155206]: -- 172.21.15.134:0/57637410 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c5d93171e0 con 0x55c5d92ed000 2023-08-11T15:57:28.722 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: starting mds.h at 2023-08-11T15:57:28.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: pgmap v342: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2023-08-11T15:57:28.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/1104446886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:57:28.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/188588884' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T15:57:28.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-11T15:57:28.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T15:57:28.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: osdmap e110: 12 total, 12 up, 12 in 2023-08-11T15:57:28.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/188588884' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-11T15:57:28.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mon[130447]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-08-11T15:57:28.724 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b[155523]: ignoring --setuser ceph since I am not root 2023-08-11T15:57:28.725 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b[155523]: ignoring --setgroup ceph since I am not root 2023-08-11T15:57:28.725 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-mds[155547]: -- 172.21.15.134:0/392581268 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557ca4f891e0 con 0x557ca4f5f000 2023-08-11T15:57:28.725 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:57:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b[155523]: starting mds.b at 2023-08-11T15:57:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:28.752+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff438110920 con 0x7ff438101230 2023-08-11T15:57:28.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: pgmap v342: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2023-08-11T15:57:28.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/1104446886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T15:57:28.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/188588884' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T15:57:28.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-11T15:57:28.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T15:57:28.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: osdmap e110: 12 total, 12 up, 12 in 2023-08-11T15:57:28.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/188588884' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-11T15:57:28.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:28 smithi192 ceph-mon[130806]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-08-11T15:57:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.249+0000 7ff43c9fd700 1 -- 172.21.15.32:0/4050150142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff42c098520 con 0x7ff438101230 2023-08-11T15:57:29.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff4200a00c0 msgr2=0x7ff4200a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff4200a00c0 0x7ff4200a2580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff42800b2e0 tx=0x7ff428005fd0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438101230 msgr2=0x7ff43819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438101230 0x7ff43819ee70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00d950 tx=0x7ff42c00dc60 comp rx=0 tx=0).stop 2023-08-11T15:57:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 shutdown_connections 2023-08-11T15:57:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7ff4200a00c0 0x7ff4200a2580 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff438106880 0x7ff4381989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff438100870 0x7ff43819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 --2- 172.21.15.32:0/4050150142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff438101230 0x7ff43819ee70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 >> 172.21.15.32:0/4050150142 conn(0x7ff4380fc3a0 msgr2=0x7ff43810bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 shutdown_connections 2023-08-11T15:57:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.252+0000 7ff43d9ff700 1 -- 172.21.15.32:0/4050150142 wait complete. 2023-08-11T15:57:29.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-11T15:57:29.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.489+0000 7fa91dbff700 1 -- 172.21.15.32:0/827835435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 msgr2=0x7fa918106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:29.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.489+0000 7fa91dbff700 1 --2- 172.21.15.32:0/827835435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa918106c60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa908009990 tx=0x7fa908009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.489+0000 7fa91dbff700 1 -- 172.21.15.32:0/827835435 shutdown_connections 2023-08-11T15:57:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.489+0000 7fa91dbff700 1 --2- 172.21.15.32:0/827835435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa918106c60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.489+0000 7fa91dbff700 1 --2- 172.21.15.32:0/827835435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa918101230 0x7fa91810d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.489+0000 7fa91dbff700 1 --2- 172.21.15.32:0/827835435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa918100870 0x7fa918100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.489+0000 7fa91dbff700 1 -- 172.21.15.32:0/827835435 >> 172.21.15.32:0/827835435 conn(0x7fa9180fc3a0 msgr2=0x7fa9180fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.490+0000 7fa91dbff700 1 -- 172.21.15.32:0/827835435 shutdown_connections 2023-08-11T15:57:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.490+0000 7fa91dbff700 1 -- 172.21.15.32:0/827835435 wait complete. 2023-08-11T15:57:29.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 Processor -- start 2023-08-11T15:57:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 -- start start 2023-08-11T15:57:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa918100870 0x7fa918072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa918101230 0x7fa918073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa9181a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918073990 con 0x7fa918106880 2023-08-11T15:57:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181a50b0 con 0x7fa918101230 2023-08-11T15:57:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.491+0000 7fa91dbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181a51f0 con 0x7fa918100870 2023-08-11T15:57:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa9181a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa916ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa918101230 0x7fa918073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:29.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa9181a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40942/0 (socket says 172.21.15.32:40942) 2023-08-11T15:57:29.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa9177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa918100870 0x7fa918072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:29.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 -- 172.21.15.32:0/115982487 learned_addr learned my addr 172.21.15.32:0/115982487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:29.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa916ffd700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa918101230 0x7fa918073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.32:36498/0 (socket says 172.21.15.32:36498) 2023-08-11T15:57:29.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 -- 172.21.15.32:0/115982487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa918100870 msgr2=0x7fa918072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:29.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa918100870 0x7fa918072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 -- 172.21.15.32:0/115982487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa918101230 msgr2=0x7fa918073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:29.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa918101230 0x7fa918073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:29.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 -- 172.21.15.32:0/115982487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080095f0 con 0x7fa918106880 2023-08-11T15:57:29.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.492+0000 7fa917fff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa9181a4b70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa91000ef30 tx=0x7fa91000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.493+0000 7fa914ff9700 1 -- 172.21.15.32:0/115982487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910018070 con 0x7fa918106880 2023-08-11T15:57:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.493+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9181a53f0 con 0x7fa918106880 2023-08-11T15:57:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.493+0000 7fa914ff9700 1 -- 172.21.15.32:0/115982487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91000f040 con 0x7fa918106880 2023-08-11T15:57:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.493+0000 7fa914ff9700 1 -- 172.21.15.32:0/115982487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000cd00 con 0x7fa918106880 2023-08-11T15:57:29.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.493+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9181a58d0 con 0x7fa918106880 2023-08-11T15:57:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.494+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9180676a0 con 0x7fa918106880 2023-08-11T15:57:29.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.496+0000 7fa914ff9700 1 -- 172.21.15.32:0/115982487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fa910003a00 con 0x7fa918106880 2023-08-11T15:57:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.497+0000 7fa914ff9700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa9000a00c0 0x7fa9000a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.497+0000 7fa914ff9700 1 -- 172.21.15.32:0/115982487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9393+0+0 (secure 0 0 0) 0x7fa9100cb500 con 0x7fa918106880 2023-08-11T15:57:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.498+0000 7fa9177fe700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa9000a00c0 0x7fa9000a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:29.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.498+0000 7fa9177fe700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa9000a00c0 0x7fa9000a2580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b2e0 tx=0x7fa908005fb0 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:29.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.501+0000 7fa914ff9700 1 -- 172.21.15.32:0/115982487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa910094c10 con 0x7fa918106880 2023-08-11T15:57:29.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4050150142' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:57:29.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-11T15:57:29.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-11T15:57:29.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: Cluster is now healthy 2023-08-11T15:57:29.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6836/3686903181,v1:172.21.15.134:6838/3686903181] up:boot 2023-08-11T15:57:29.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: mds.? [v2:172.21.15.134:6837/2219330613,v1:172.21.15.134:6839/2219330613] up:boot 2023-08-11T15:57:29.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4050150142' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-11T15:57:29.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: fsmap 12 up:standby 2023-08-11T15:57:29.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:57:29.555 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:29 smithi032 ceph-mon[97103]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:57:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:29.712+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa9181108e0 con 0x7fa918106880 2023-08-11T15:57:29.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4050150142' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:57:29.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-11T15:57:29.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-11T15:57:29.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: Cluster is now healthy 2023-08-11T15:57:29.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6836/3686903181,v1:172.21.15.134:6838/3686903181] up:boot 2023-08-11T15:57:29.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: mds.? [v2:172.21.15.134:6837/2219330613,v1:172.21.15.134:6839/2219330613] up:boot 2023-08-11T15:57:29.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4050150142' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-11T15:57:29.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: fsmap 12 up:standby 2023-08-11T15:57:29.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:57:29.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:29 smithi134 ceph-mon[130447]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:57:29.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4050150142' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T15:57:29.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-11T15:57:29.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-11T15:57:29.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: Cluster is now healthy 2023-08-11T15:57:29.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6836/3686903181,v1:172.21.15.134:6838/3686903181] up:boot 2023-08-11T15:57:29.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: mds.? [v2:172.21.15.134:6837/2219330613,v1:172.21.15.134:6839/2219330613] up:boot 2023-08-11T15:57:29.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4050150142' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-11T15:57:29.837 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: fsmap 12 up:standby 2023-08-11T15:57:29.837 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T15:57:29.838 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:29 smithi192 ceph-mon[130806]: from='mgr.15315 172.21.15.32:0/1108237796' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T15:57:30.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.420+0000 7fa914ff9700 1 -- 172.21.15.32:0/115982487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa910098230 con 0x7fa918106880 2023-08-11T15:57:30.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.429+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa9000a00c0 msgr2=0x7fa9000a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:30.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.429+0000 7fa91dbff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa9000a00c0 0x7fa9000a2580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b2e0 tx=0x7fa908005fb0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.429+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 msgr2=0x7fa9181a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.429+0000 7fa91dbff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa9181a4b70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa91000ef30 tx=0x7fa91000c3a0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.430+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 shutdown_connections 2023-08-11T15:57:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.430+0000 7fa91dbff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fa9000a00c0 0x7fa9000a2580 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.430+0000 7fa91dbff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa918106880 0x7fa9181a4b70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.430+0000 7fa91dbff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa918100870 0x7fa918072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.430+0000 7fa91dbff700 1 --2- 172.21.15.32:0/115982487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa918101230 0x7fa918073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.430+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 >> 172.21.15.32:0/115982487 conn(0x7fa9180fc3a0 msgr2=0x7fa9180fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.431+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 shutdown_connections 2023-08-11T15:57:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.431+0000 7fa91dbff700 1 -- 172.21.15.32:0/115982487 wait complete. 2023-08-11T15:57:30.436 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-08-11T15:57:30.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-11T15:57:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 -- 172.21.15.32:0/2059989548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74540735f0 msgr2=0x7f745410cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 --2- 172.21.15.32:0/2059989548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74540735f0 0x7f745410cc20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7448009990 tx=0x7f7448009ca0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 -- 172.21.15.32:0/2059989548 shutdown_connections 2023-08-11T15:57:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 --2- 172.21.15.32:0/2059989548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f745410d160 0x7f745410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 --2- 172.21.15.32:0/2059989548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7454072cd0 0x7f74540730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 --2- 172.21.15.32:0/2059989548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74540735f0 0x7f745410cc20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 -- 172.21.15.32:0/2059989548 >> 172.21.15.32:0/2059989548 conn(0x7f74540fc390 msgr2=0x7f74540fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 -- 172.21.15.32:0/2059989548 shutdown_connections 2023-08-11T15:57:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.674+0000 7f745a7ff700 1 -- 172.21.15.32:0/2059989548 wait complete. 2023-08-11T15:57:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.675+0000 7f745a7ff700 1 Processor -- start 2023-08-11T15:57:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.675+0000 7f745a7ff700 1 -- start start 2023-08-11T15:57:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f745a7ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7454072cd0 0x7f745419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f745a7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74540735f0 0x7f745419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f745a7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745410d160 0x7f7454198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f745a7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74541990d0 con 0x7f745410d160 2023-08-11T15:57:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f745a7ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7454199240 con 0x7f7454072cd0 2023-08-11T15:57:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f745a7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7454199400 con 0x7f74540735f0 2023-08-11T15:57:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745410d160 0x7f7454198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745410d160 0x7f7454198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40964/0 (socket says 172.21.15.32:40964) 2023-08-11T15:57:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7453fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7454072cd0 0x7f745419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 -- 172.21.15.32:0/4264566462 learned_addr learned my addr 172.21.15.32:0/4264566462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f74537fe700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74540735f0 0x7f745419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 -- 172.21.15.32:0/4264566462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74540735f0 msgr2=0x7f745419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74540735f0 0x7f745419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 -- 172.21.15.32:0/4264566462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7454072cd0 msgr2=0x7f745419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7454072cd0 0x7f745419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.676+0000 7f7458d9c700 1 -- 172.21.15.32:0/4264566462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74480095f0 con 0x7f745410d160 2023-08-11T15:57:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.677+0000 7f7458d9c700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745410d160 0x7f7454198a60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f744c00ef30 tx=0x7f744c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.677+0000 7f74517fa700 1 -- 172.21.15.32:0/4264566462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c018070 con 0x7f745410d160 2023-08-11T15:57:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.677+0000 7f74517fa700 1 -- 172.21.15.32:0/4264566462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f744c00f040 con 0x7f745410d160 2023-08-11T15:57:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.677+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7454199680 con 0x7f745410d160 2023-08-11T15:57:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.677+0000 7f74517fa700 1 -- 172.21.15.32:0/4264566462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c00cd00 con 0x7f745410d160 2023-08-11T15:57:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.677+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74541a59e0 con 0x7f745410d160 2023-08-11T15:57:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.678+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74540676a0 con 0x7f745410d160 2023-08-11T15:57:30.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.680+0000 7f74517fa700 1 -- 172.21.15.32:0/4264566462 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7f744c003a00 con 0x7f745410d160 2023-08-11T15:57:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.681+0000 7f74517fa700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f743c0a00c0 0x7f743c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.681+0000 7f74517fa700 1 -- 172.21.15.32:0/4264566462 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8868+0+0 (secure 0 0 0) 0x7f744c0cb2f0 con 0x7f745410d160 2023-08-11T15:57:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.681+0000 7f7453fff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f743c0a00c0 0x7f743c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:30.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.682+0000 7f7453fff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f743c0a00c0 0x7f743c0a2580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7444009c60 tx=0x7f7444009260 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:30.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.685+0000 7f74517fa700 1 -- 172.21.15.32:0/4264566462 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f744c094c10 con 0x7f745410d160 2023-08-11T15:57:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:30 smithi134 ceph-mon[130447]: pgmap v344: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2023-08-11T15:57:30.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:30 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/115982487' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:30.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:30 smithi032 ceph-mon[97103]: pgmap v344: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2023-08-11T15:57:30.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:30 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/115982487' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:30.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:30 smithi192 ceph-mon[130806]: pgmap v344: 129 pgs: 129 active+clean; 55 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2023-08-11T15:57:30.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:30 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/115982487' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:30.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:30.901+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7454199f70 con 0x7f745410d160 2023-08-11T15:57:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.435+0000 7f74517fa700 1 -- 172.21.15.32:0/4264566462 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f744c098230 con 0x7f745410d160 2023-08-11T15:57:31.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.442+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f743c0a00c0 msgr2=0x7f743c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.442+0000 7f745a7ff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f743c0a00c0 0x7f743c0a2580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7444009c60 tx=0x7f7444009260 comp rx=0 tx=0).stop 2023-08-11T15:57:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.443+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745410d160 msgr2=0x7f7454198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.443+0000 7f745a7ff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745410d160 0x7f7454198a60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f744c00ef30 tx=0x7f744c00c3a0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 shutdown_connections 2023-08-11T15:57:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7f743c0a00c0 0x7f743c0a2580 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745410d160 0x7f7454198a60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7454072cd0 0x7f745419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 --2- 172.21.15.32:0/4264566462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74540735f0 0x7f745419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 >> 172.21.15.32:0/4264566462 conn(0x7f74540fc390 msgr2=0x7f745410f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 shutdown_connections 2023-08-11T15:57:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.446+0000 7f745a7ff700 1 -- 172.21.15.32:0/4264566462 wait complete. 2023-08-11T15:57:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-08-11T15:57:31.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-08-11T15:57:31.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 -- 172.21.15.32:0/803429861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470073670 msgr2=0x7fd47010cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 --2- 172.21.15.32:0/803429861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470073670 0x7fd47010cb70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd4640098f0 tx=0x7fd464009c00 comp rx=0 tx=0).stop 2023-08-11T15:57:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 -- 172.21.15.32:0/803429861 shutdown_connections 2023-08-11T15:57:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 --2- 172.21.15.32:0/803429861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd47010d0b0 0x7fd47010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 --2- 172.21.15.32:0/803429861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd470072d50 0x7fd470073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 --2- 172.21.15.32:0/803429861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470073670 0x7fd47010cb70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 -- 172.21.15.32:0/803429861 >> 172.21.15.32:0/803429861 conn(0x7fd4700fc3a0 msgr2=0x7fd4700fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 -- 172.21.15.32:0/803429861 shutdown_connections 2023-08-11T15:57:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.688+0000 7fd4769ff700 1 -- 172.21.15.32:0/803429861 wait complete. 2023-08-11T15:57:31.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.689+0000 7fd4769ff700 1 Processor -- start 2023-08-11T15:57:31.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.689+0000 7fd4769ff700 1 -- start start 2023-08-11T15:57:31.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd4769ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470072d50 0x7fd47019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:31.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd4769ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd470073670 0x7fd47019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:31.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd4769ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd47010d0b0 0x7fd470198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:31.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd4769ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701990c0 con 0x7fd470072d50 2023-08-11T15:57:31.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd4769ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd470199230 con 0x7fd470073670 2023-08-11T15:57:31.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd4769ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701993f0 con 0x7fd47010d0b0 2023-08-11T15:57:31.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd46ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470072d50 0x7fd47019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:31.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd46f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd470073670 0x7fd47019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:31.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd474f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd47010d0b0 0x7fd470198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:31.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd46ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470072d50 0x7fd47019e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40974/0 (socket says 172.21.15.32:40974) 2023-08-11T15:57:31.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.690+0000 7fd46ffff700 1 -- 172.21.15.32:0/514993430 learned_addr learned my addr 172.21.15.32:0/514993430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-08-11T15:57:31.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46f7fe700 1 -- 172.21.15.32:0/514993430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd47010d0b0 msgr2=0x7fd470198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:31.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46f7fe700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd47010d0b0 0x7fd470198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46f7fe700 1 -- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470072d50 msgr2=0x7fd47019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:31.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46f7fe700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470072d50 0x7fd47019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:31.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46f7fe700 1 -- 172.21.15.32:0/514993430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd464009520 con 0x7fd470073670 2023-08-11T15:57:31.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46ffff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470072d50 0x7fd47019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T15:57:31.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46f7fe700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd470073670 0x7fd47019ef20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd464009810 tx=0x7fd46400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:31.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd46d7fa700 1 -- 172.21.15.32:0/514993430 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46401c070 con 0x7fd470073670 2023-08-11T15:57:31.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.691+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd470199670 con 0x7fd470073670 2023-08-11T15:57:31.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.692+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4701a59d0 con 0x7fd470073670 2023-08-11T15:57:31.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.692+0000 7fd46d7fa700 1 -- 172.21.15.32:0/514993430 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd46400bba0 con 0x7fd470073670 2023-08-11T15:57:31.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.692+0000 7fd46d7fa700 1 -- 172.21.15.32:0/514993430 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640038e0 con 0x7fd470073670 2023-08-11T15:57:31.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.692+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4700676a0 con 0x7fd470073670 2023-08-11T15:57:31.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.695+0000 7fd46d7fa700 1 -- 172.21.15.32:0/514993430 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 134802+0+0 (secure 0 0 0) 0x7fd464020030 con 0x7fd470073670 2023-08-11T15:57:31.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.696+0000 7fd46d7fa700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd4580a00c0 0x7fd4580a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T15:57:31.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.696+0000 7fd46d7fa700 1 -- 172.21.15.32:0/514993430 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8338+0+0 (secure 0 0 0) 0x7fd4640cc680 con 0x7fd470073670 2023-08-11T15:57:31.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.696+0000 7fd46ffff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd4580a00c0 0x7fd4580a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T15:57:31.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.697+0000 7fd46ffff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd4580a00c0 0x7fd4580a2580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd460005950 tx=0x7fd460009310 comp rx=0 tx=0).ready entity=mgr.15315 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T15:57:31.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.698+0000 7fd46d7fa700 1 -- 172.21.15.32:0/514993430 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd4640961b0 con 0x7fd470073670 2023-08-11T15:57:31.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:31 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/115982487' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T15:57:31.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:31 smithi134 ceph-mon[130447]: osdmap e111: 12 total, 12 up, 12 in 2023-08-11T15:57:31.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:31 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4264566462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:31.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:31 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/115982487' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T15:57:31.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:31 smithi032 ceph-mon[97103]: osdmap e111: 12 total, 12 up, 12 in 2023-08-11T15:57:31.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:31 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4264566462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:31.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:31 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/115982487' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T15:57:31.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:31 smithi192 ceph-mon[130806]: osdmap e111: 12 total, 12 up, 12 in 2023-08-11T15:57:31.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:31 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4264566462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:31.918+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd470199f60 con 0x7fd470073670 2023-08-11T15:57:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.438+0000 7fd46d7fa700 1 -- 172.21.15.32:0/514993430 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v113) v1 ==== 178+0+0 (secure 0 0 0) 0x7fd4640997d0 con 0x7fd470073670 2023-08-11T15:57:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.445+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd4580a00c0 msgr2=0x7fd4580a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.445+0000 7fd4769ff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd4580a00c0 0x7fd4580a2580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd460005950 tx=0x7fd460009310 comp rx=0 tx=0).stop 2023-08-11T15:57:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.445+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd470073670 msgr2=0x7fd47019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T15:57:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.445+0000 7fd4769ff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd470073670 0x7fd47019ef20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd464009810 tx=0x7fd46400fa00 comp rx=0 tx=0).stop 2023-08-11T15:57:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 shutdown_connections 2023-08-11T15:57:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:6800/1148710426,v1:172.21.15.32:6801/1148710426] conn(0x7fd4580a00c0 0x7fd4580a2580 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470072d50 0x7fd47019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd47010d0b0 0x7fd470198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 --2- 172.21.15.32:0/514993430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd470073670 0x7fd47019ef20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T15:57:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 >> 172.21.15.32:0/514993430 conn(0x7fd4700fc3a0 msgr2=0x7fd47010f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T15:57:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 shutdown_connections 2023-08-11T15:57:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-08-11T15:57:32.449+0000 7fd4769ff700 1 -- 172.21.15.32:0/514993430 wait complete. 2023-08-11T15:57:32.456 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data_ec' removed 2023-08-11T15:57:32.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T15:57:32.554 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T15:57:32.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T15:57:32.628 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-08-11T15:57:32.628 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-11T15:57:32.672 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-11T15:57:32.690 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-11T15:57:32.712 INFO:tasks.cephadm:Stopping all daemons... 2023-08-11T15:57:32.713 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-08-11T15:57:32.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a 2023-08-11T15:57:32.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:32 smithi134 ceph-mon[130447]: pgmap v346: 97 pgs: 97 active+clean; 46 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:57:32.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:32 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/4264566462' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T15:57:32.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:32 smithi134 ceph-mon[130447]: osdmap e112: 12 total, 12 up, 12 in 2023-08-11T15:57:32.721 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:32 smithi134 ceph-mon[130447]: from='client.? 172.21.15.32:0/514993430' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:32.722 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:32 smithi134 ceph-mon[130447]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:32.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:32 smithi032 ceph-mon[97103]: pgmap v346: 97 pgs: 97 active+clean; 46 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:57:32.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:32 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/4264566462' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T15:57:32.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:32 smithi032 ceph-mon[97103]: osdmap e112: 12 total, 12 up, 12 in 2023-08-11T15:57:32.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:32 smithi032 ceph-mon[97103]: from='client.? 172.21.15.32:0/514993430' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:32.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:32 smithi032 ceph-mon[97103]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:32.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:32 smithi192 ceph-mon[130806]: pgmap v346: 97 pgs: 97 active+clean; 46 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T15:57:32.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:32 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/4264566462' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T15:57:32.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:32 smithi192 ceph-mon[130806]: osdmap e112: 12 total, 12 up, 12 in 2023-08-11T15:57:32.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:32 smithi192 ceph-mon[130806]: from='client.? 172.21.15.32:0/514993430' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:32.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:32 smithi192 ceph-mon[130806]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T15:57:33.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:32 smithi032 systemd[1]: Stopping Ceph mon.a for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:33.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-a[97079]: 2023-08-11T15:57:33.240+0000 7f9743f58700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:57:33.554 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-a[97079]: 2023-08-11T15:57:33.241+0000 7f9743f58700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-08-11T15:57:33.973 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 11 15:57:33 smithi032 bash[154773]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-a 2023-08-11T15:57:34.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.a.service' 2023-08-11T15:57:34.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:34.128 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-08-11T15:57:34.128 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-08-11T15:57:34.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.b 2023-08-11T15:57:34.471 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:34 smithi134 systemd[1]: Stopping Ceph mon.b for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-b[130423]: 2023-08-11T15:57:34.526+0000 7f94c7733700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:57:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-b[130423]: 2023-08-11T15:57:34.526+0000 7f94c7733700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-08-11T15:57:35.083 INFO:journalctl@ceph.mon.b.smithi134.stdout:Aug 11 15:57:35 smithi134 bash[161585]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-b 2023-08-11T15:57:35.278 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.b.service' 2023-08-11T15:57:35.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:35.334 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-08-11T15:57:35.335 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-08-11T15:57:35.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.c 2023-08-11T15:57:35.758 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:35 smithi192 systemd[1]: Stopping Ceph mon.c for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:36.055 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-c[130782]: 2023-08-11T15:57:35.755+0000 7f29a167a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:57:36.056 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-c[130782]: 2023-08-11T15:57:35.755+0000 7f29a167a700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-08-11T15:57:36.333 INFO:journalctl@ceph.mon.c.smithi192.stdout:Aug 11 15:57:36 smithi192 bash[162566]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mon-c 2023-08-11T15:57:36.407 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mon.c.service' 2023-08-11T15:57:36.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:36.455 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-08-11T15:57:36.455 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-08-11T15:57:36.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.x 2023-08-11T15:57:36.804 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:57:36 smithi032 systemd[1]: Stopping Ceph mgr.x for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:37.263 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:57:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x[97353]: 2023-08-11T15:57:36.974+0000 7f340ea60700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:57:37.553 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 11 15:57:37 smithi032 bash[154914]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-x 2023-08-11T15:57:37.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.x.service' 2023-08-11T15:57:37.843 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:37.843 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-08-11T15:57:37.844 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-08-11T15:57:37.844 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.y 2023-08-11T15:57:38.221 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:57:37 smithi134 systemd[1]: Stopping Ceph mgr.y for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:38.555 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:57:38 smithi134 bash[161724]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-y 2023-08-11T15:57:38.737 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Aug 11 15:57:38 smithi134 systemd[1]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-08-11T15:57:38.742 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.y.service' 2023-08-11T15:57:38.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:38.782 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-08-11T15:57:38.782 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-08-11T15:57:38.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.z 2023-08-11T15:57:39.083 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:57:38 smithi192 systemd[1]: Stopping Ceph mgr.z for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:39.491 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:57:39 smithi192 bash[162717]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mgr-z 2023-08-11T15:57:39.670 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Aug 11 15:57:39 smithi192 systemd[1]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-08-11T15:57:39.673 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mgr.z.service' 2023-08-11T15:57:39.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:39.711 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-08-11T15:57:39.711 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-08-11T15:57:39.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.0 2023-08-11T15:57:40.053 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:57:39 smithi032 systemd[1]: Stopping Ceph osd.0 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:40.553 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:57:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-0[109333]: 2023-08-11T15:57:40.240+0000 7f285f82c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:57:40.553 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:57:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-0[109333]: 2023-08-11T15:57:40.240+0000 7f285f82c700 -1 osd.0 113 *** Got signal Terminated *** 2023-08-11T15:57:40.554 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:57:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-0[109333]: 2023-08-11T15:57:40.240+0000 7f285f82c700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:57:46.144 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 11 15:57:46 smithi032 bash[155061]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-0 2023-08-11T15:57:47.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.0.service' 2023-08-11T15:57:47.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:47.981 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-08-11T15:57:47.981 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-08-11T15:57:47.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.1 2023-08-11T15:57:48.343 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:57:48 smithi134 systemd[1]: Stopping Ceph osd.1 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:48.721 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:57:48 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-1[135518]: 2023-08-11T15:57:48.340+0000 7f56c2e73700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:57:48.721 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:57:48 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-1[135518]: 2023-08-11T15:57:48.340+0000 7f56c2e73700 -1 osd.1 113 *** Got signal Terminated *** 2023-08-11T15:57:48.722 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:57:48 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-1[135518]: 2023-08-11T15:57:48.340+0000 7f56c2e73700 -1 osd.1 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:57:54.119 INFO:journalctl@ceph.osd.1.smithi134.stdout:Aug 11 15:57:54 smithi134 bash[161878]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-1 2023-08-11T15:57:55.384 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.1.service' 2023-08-11T15:57:55.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:57:55.426 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-08-11T15:57:55.426 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-08-11T15:57:55.427 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.2 2023-08-11T15:57:55.780 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:57:55 smithi192 systemd[1]: Stopping Ceph osd.2 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:57:56.083 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:57:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-2[136609]: 2023-08-11T15:57:55.777+0000 7feeef0ef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:57:56.084 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:57:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-2[136609]: 2023-08-11T15:57:55.777+0000 7feeef0ef700 -1 osd.2 113 *** Got signal Terminated *** 2023-08-11T15:57:56.084 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:57:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-2[136609]: 2023-08-11T15:57:55.777+0000 7feeef0ef700 -1 osd.2 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:01.566 INFO:journalctl@ceph.osd.2.smithi192.stdout:Aug 11 15:58:01 smithi192 bash[162870]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-2 2023-08-11T15:58:02.730 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.2.service' 2023-08-11T15:58:02.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:02.772 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-08-11T15:58:02.772 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-08-11T15:58:02.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.3 2023-08-11T15:58:03.298 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 11 15:58:02 smithi032 systemd[1]: Stopping Ceph osd.3 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:03.553 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 11 15:58:03 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-3[114810]: 2023-08-11T15:58:03.295+0000 7f97da55e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:03.553 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 11 15:58:03 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-3[114810]: 2023-08-11T15:58:03.295+0000 7f97da55e700 -1 osd.3 113 *** Got signal Terminated *** 2023-08-11T15:58:03.553 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 11 15:58:03 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-3[114810]: 2023-08-11T15:58:03.295+0000 7f97da55e700 -1 osd.3 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:06.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:06 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:06.362+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:07.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:07 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:07.404+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:08.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:07.703+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:08.396 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:08 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:08.071+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:08.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:08 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:08.395+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:09.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:08.744+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:09.438 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 11 15:58:09 smithi032 bash[155342]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-3 2023-08-11T15:58:09.439 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:09 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:09.080+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:09.804 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:09 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:09.436+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:10.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:09.716+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:10.484 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:10 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:10.077+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:10.766 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:10 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:10.481+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:11.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:10.686+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:11.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.3.service' 2023-08-11T15:58:11.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:11.288 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-08-11T15:58:11.288 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-08-11T15:58:11.289 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.4 2023-08-11T15:58:11.478 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:11 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:11.064+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:11.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:11 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:11.288+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:11.673 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:58:11 smithi134 systemd[1]: Stopping Ceph osd.4 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:11.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:11 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:11.475+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:11.971 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:58:11 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-4[140915]: 2023-08-11T15:58:11.670+0000 7fc70cc34700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:11.971 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:58:11 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-4[140915]: 2023-08-11T15:58:11.670+0000 7fc70cc34700 -1 osd.4 113 *** Got signal Terminated *** 2023-08-11T15:58:11.972 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:58:11 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-4[140915]: 2023-08-11T15:58:11.670+0000 7fc70cc34700 -1 osd.4 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:12.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:11 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:11.678+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:12.303 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:12 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:12.017+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:12.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:12 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:12.285+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:12.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:12 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:12.457+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:13.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:12 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:12.702+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:13.440 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:13 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:13.056+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:13.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:13 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:13.261+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:13.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:13 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:13.438+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:14.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:13 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:13.674+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:14.458 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:14 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:14.081+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:14.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:14 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:14.243+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:14.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:14 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:14.448+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:14.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:14 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:14.457+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:15.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:14 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:14.714+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:15.442 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:15 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:15.056+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:15.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:15 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:15.223+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:15.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:15 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:15.431+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:15.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:15 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:15.235+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:15.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:15 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:15.440+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:15.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:15 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:15.440+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:16.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:15 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:15.749+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:16.084 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:15 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:15.582+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:16.450 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:16 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:16.097+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:16.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:16 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:16.268+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:16.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:16 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:16.237+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:16.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:16 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:16.399+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:16.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:16 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:16.448+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:16.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:16 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:16.448+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:17.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:16 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:16.768+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:17.084 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:16 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:16.583+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:17.428 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:17 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:17.142+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:17.428 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:17 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:17.142+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:17.546 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:17 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:17.281+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:17.721 INFO:journalctl@ceph.osd.4.smithi134.stdout:Aug 11 15:58:17 smithi134 bash[162148]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-4 2023-08-11T15:58:17.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:17 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:17.267+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:17.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:17 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:17.447+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:17.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:17 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:17.426+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:17.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:17 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:17.426+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:17.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:17 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:17.545+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:17.834 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:17 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:17.792+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:17.834 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:17 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:17.792+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.677780+0000 front 2023-08-11T15:57:52.677871+0000 (oldest deadline 2023-08-11T15:58:17.377822+0000) 2023-08-11T15:58:18.408 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:18 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:18.317+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:18.475 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:18.159+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:18.476 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:18.159+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:18.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:18 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:18.517+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:18.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:18 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:18.288+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:18.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:18 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:18.405+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:18.786 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.4.service' 2023-08-11T15:58:18.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:18.473+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:18.804 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:18.473+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:18.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:18.826 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-08-11T15:58:18.827 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-08-11T15:58:18.827 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.5 2023-08-11T15:58:19.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:18 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:18.816+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:19.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:18 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:18.816+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.677780+0000 front 2023-08-11T15:57:52.677871+0000 (oldest deadline 2023-08-11T15:58:17.377822+0000) 2023-08-11T15:58:19.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:18 smithi192 systemd[1]: Stopping Ceph osd.5 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:19.521 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:19 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:19.153+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:19.521 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:19 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:19.153+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:19.541 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:19 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:19.311+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:19.542 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:19 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:19.174+0000 7f7acf94c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:19.542 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:19 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:19.174+0000 7f7acf94c700 -1 osd.5 113 *** Got signal Terminated *** 2023-08-11T15:58:19.543 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:19 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:19.174+0000 7f7acf94c700 -1 osd.5 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:19.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:19 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:19.275+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:19.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:19 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:19.415+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:19.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:19 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:19.519+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:19.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:19 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:19.519+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:19.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:19 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:19.540+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:20.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:19 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:19.807+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:20.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:19 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:19.808+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.677780+0000 front 2023-08-11T15:57:52.677871+0000 (oldest deadline 2023-08-11T15:58:17.377822+0000) 2023-08-11T15:58:20.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:20.133+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:20.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:20.133+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:20.569 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:20 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:20.311+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:20.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:20 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:20.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:20.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:20 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:20.265+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:20.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:20 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:20.567+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:20.834 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:20 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:20.771+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:20.834 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:20 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:20.771+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.677780+0000 front 2023-08-11T15:57:52.677871+0000 (oldest deadline 2023-08-11T15:58:17.377822+0000) 2023-08-11T15:58:21.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:20.553+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:21.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:20 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:20.553+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:21.520 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:21 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:21.267+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:21.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:21 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:21.138+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:21.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:21 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:21.138+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:21.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:21 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:21.258+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:21.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:21 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:21.430+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:21.798 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:21 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:21.518+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:22.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:21 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:21.551+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:22.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:21 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:21.551+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:22.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:21 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:21.796+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:22.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:21 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:21.796+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.677780+0000 front 2023-08-11T15:57:52.677871+0000 (oldest deadline 2023-08-11T15:58:17.377822+0000) 2023-08-11T15:58:22.506 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:22.154+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:22.506 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:22.154+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:22.507 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:22.154+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:22.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:22.533+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:22.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:22.292+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:22.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:22.292+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:22.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:22 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:22.228+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:22.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:22 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:22.401+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:22.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:22 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:22.401+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:22.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:22.504+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:22.804 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:22.504+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:22.804 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:22 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:22.504+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:23.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:22.811+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:23.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:22.811+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.677780+0000 front 2023-08-11T15:57:52.677871+0000 (oldest deadline 2023-08-11T15:58:17.377822+0000) 2023-08-11T15:58:23.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:22 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:22.811+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:57.378317+0000 front 2023-08-11T15:57:57.378345+0000 (oldest deadline 2023-08-11T15:58:22.678397+0000) 2023-08-11T15:58:23.544 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:23.134+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:23.544 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:23.134+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:23.545 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:23.134+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:23.720 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:23 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:23.256+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:23.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:23 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:23.256+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:23.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:23 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:23.364+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:23.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:23 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:23.364+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:23.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:23.542+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:23.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:23.542+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:23.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:23.542+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:23.804 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:23 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:23.516+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:23.804 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:23 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:23.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:23.805 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:23 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:23.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:24.083 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:23 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:23.803+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:43.276796+0000 front 2023-08-11T15:57:43.277010+0000 (oldest deadline 2023-08-11T15:58:06.776688+0000) 2023-08-11T15:58:24.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:23 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:23.803+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.677780+0000 front 2023-08-11T15:57:52.677871+0000 (oldest deadline 2023-08-11T15:58:17.377822+0000) 2023-08-11T15:58:24.084 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:23 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5[141965]: 2023-08-11T15:58:23.803+0000 7f7aca73f700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:57.378317+0000 front 2023-08-11T15:57:57.378345+0000 (oldest deadline 2023-08-11T15:58:22.678397+0000) 2023-08-11T15:58:24.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:24.148+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:24.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:24.148+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:24.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:24.148+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:24.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:24 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:24.251+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:24.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:24 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:24.251+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:24.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:24 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:24.336+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:24.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:24 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:24.336+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:24.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:24 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:24.471+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:24.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:24 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:24.471+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:24.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:24 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:24.335+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:24.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:24 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:24.335+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:25.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:24.591+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:25.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:24.591+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:25.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:24 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:24.591+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:25.317 INFO:journalctl@ceph.osd.5.smithi192.stdout:Aug 11 15:58:24 smithi192 bash[163132]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-5 2023-08-11T15:58:25.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:25.134+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:25.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:25.134+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:25.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:25.134+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:25.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:25 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:25.433+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:25.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:25 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:25.433+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:25.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:25 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:25.315+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:25.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:25 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:25.315+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:25.720 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:25 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:25.250+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:25.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:25 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:25.250+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:25.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:25 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:25.309+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:25.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:25 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:25.309+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:26.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:25.589+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:26.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:25.589+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:26.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:25 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:25.589+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:26.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.5.service' 2023-08-11T15:58:26.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:26.394 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-08-11T15:58:26.395 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-08-11T15:58:26.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.6 2023-08-11T15:58:26.464 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:26.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:26.464 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:26.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:26.464 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:26.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:26.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:26 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:26.263+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:26.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:26 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:26.263+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:26.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:26 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:26.316+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:26.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:26 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:26.316+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:26.803 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:26 smithi032 systemd[1]: Stopping Ceph osd.6 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:26.804 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:26.629+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:26.804 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:26.629+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:26.805 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:26.629+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:26.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:26 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:26.424+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:26.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:26 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:26.424+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:26.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:26 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:26.354+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:26.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:26 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:26.354+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:27.206 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:26.900+0000 7fdc0bdae700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:27.206 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:26.900+0000 7fdc0bdae700 -1 osd.6 113 *** Got signal Terminated *** 2023-08-11T15:58:27.207 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:26 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:26.900+0000 7fdc0bdae700 -1 osd.6 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:27.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:27 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:27.203+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:27.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:27 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:27.203+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:27.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:27 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:27.203+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:27.720 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:27 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:27.294+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:27.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:27 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:27.294+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:27.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:27 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:27.343+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:27.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:27 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:27.343+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:27.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:27 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:27.414+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:27.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:27 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:27.414+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:27.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:27 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:27.370+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:27.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:27 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:27.370+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:28.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:27 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:27.662+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:28.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:27 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:27.662+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:28.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:27 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:27.662+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:28.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:28 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:28.220+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:28.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:28 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:28.220+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:28.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:28 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:28.220+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:28.720 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:28.314+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:28.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:28.314+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:28.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:28.382+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:28.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:28 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:28.382+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:28.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:28 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:28.370+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:28.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:28 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:28.370+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:28.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:28 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:28.375+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:28.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:28 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:28.375+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:29.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:28 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:28.666+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:29.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:28 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:28.666+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:29.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:28 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:28.666+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:29.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:29 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:29.221+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:29.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:29 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:29.221+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:29.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:29 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:29.221+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:29.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:29 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:29.409+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:29.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:29 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:29.409+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:29.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:29 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:29.332+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:29.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:29 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:29.332+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:29.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:29 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:29.332+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:29.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:29 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:29.386+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:29.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:29 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:29.386+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:29.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:29 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:29.386+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:29.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:29 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:29.386+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:29.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:29 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:29.386+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:29.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:29 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:29.386+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:30.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:29 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:29.631+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:30.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:29 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:29.631+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:30.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:29 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:29.631+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:30.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:30 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:30.216+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:30.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:30 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:30.216+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:30.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:30 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:30.216+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:30.720 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:30 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:30.319+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:30.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:30 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:30.319+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:30.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:30 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:30.319+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:30.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:30 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:30.435+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:30.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:30 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:30.435+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:30.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:30 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:30.435+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:30.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:30.412+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:30.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:30.412+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:30.835 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:30.412+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:30.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:30.368+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:30.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:30.368+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:30.836 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:30 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:30.368+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:31.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:30 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:30.620+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:31.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:30 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:30.620+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:31.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:30 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:30.620+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:31.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:31 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:31.214+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:31.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:31 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:31.214+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:31.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:31 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:31.214+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:31.556 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:31 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:31.214+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:31.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:31 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:31.411+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:31.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:31 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:31.411+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:31.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:31 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:31.411+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:31.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:31 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:31.316+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:31.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:31 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:31.316+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:31.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:31 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:31.316+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:31.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:31 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:31.397+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:31.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:31 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:31.397+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:31.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:31 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:31.397+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:31.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:31 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:31.387+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:31.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:31 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:31.387+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:31.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:31 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:31.387+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:32.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:31 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:31.598+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:40.695186+0000 front 2023-08-11T15:57:40.695212+0000 (oldest deadline 2023-08-11T15:58:05.995330+0000) 2023-08-11T15:58:32.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:31 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:31.598+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.896577+0000 front 2023-08-11T15:57:51.896501+0000 (oldest deadline 2023-08-11T15:58:15.396599+0000) 2023-08-11T15:58:32.054 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:31 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6[120227]: 2023-08-11T15:58:31.598+0000 7fdc0439c700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.297908+0000 front 2023-08-11T15:57:58.297883+0000 (oldest deadline 2023-08-11T15:58:21.798002+0000) 2023-08-11T15:58:32.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:32.223+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:32.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:32.223+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:32.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:32.223+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:32.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:32 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:32.223+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:32.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:32.341+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:32.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:32.341+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:32.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:32.341+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:32.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:32.426+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:32.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:32.426+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:32.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:32.426+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:32.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:32.370+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:32.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:32.370+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:32.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:32.370+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:32.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:32.384+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:32.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:32.384+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:32.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:32 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:32.384+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:33.053 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 11 15:58:32 smithi032 bash[155621]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-6 2023-08-11T15:58:33.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:33.206+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:33.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:33.206+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:33.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:33.206+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:33.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:33 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:33.206+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:33.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:33.329+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:33.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:33.329+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:33.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:33.329+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:33.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:33.338+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:33.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:33.338+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:33.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:33 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:33.338+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:33.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:33.337+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:33.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:33.337+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:33.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:33.337+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:33.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:33.456+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:33.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:33.456+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:33.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:33 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:33.456+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:34.476 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:34.160+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:34.476 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:34.160+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:34.477 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:34.160+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:34.477 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:34 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:34.160+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:34.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.6.service' 2023-08-11T15:58:34.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:34.323+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:34.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:34.323+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:34.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:34.323+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:34.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:34.336+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:34.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:34.336+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:34.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:34 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:34.336+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:34.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:34.589 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-08-11T15:58:34.589 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-08-11T15:58:34.590 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.7 2023-08-11T15:58:34.621 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:34.497+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:34.621 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:34.497+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:34.622 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:34.497+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:34.622 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:34.369+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:34.622 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:34.369+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:34.622 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:34.369+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:34.935 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:34 smithi134 systemd[1]: Stopping Ceph osd.7 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:34.935 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:34.933+0000 7f290c798700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:35.220 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:34.933+0000 7f290c798700 -1 osd.7 113 *** Got signal Terminated *** 2023-08-11T15:58:35.221 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:34 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:34.933+0000 7f290c798700 -1 osd.7 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:35.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:35 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:35.118+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:35.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:35 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:35.118+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:35.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:35 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:35.118+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:35.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:35 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:35.118+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:35.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:35.356+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:35.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:35.356+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:35.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:35.356+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:35.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:35.304+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:35.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:35.304+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:35.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:35 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:35.304+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:35.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:35.350+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:35.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:35.350+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:35.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:35.350+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:35.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:35.540+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:35.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:35.540+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:35.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:35 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:35.540+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:36.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:36.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:36.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:36.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:36.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:36.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:36.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:36 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:36.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:36.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:36.405+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:36.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:36.405+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:36.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:36.406+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:36.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:36.291+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:36.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:36.291+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:36.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:36 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:36.291+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:36.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:36.311+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:36.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:36.312+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:36.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:36.312+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:36.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:36.532+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:36.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:36.532+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:36.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:36 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:36.532+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:37.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:37 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:37.191+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:37.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:37 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:37.191+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:37.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:37 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:37.191+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:37.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:37 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:37.191+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:37.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:37.258+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:37.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:37.258+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:37.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:37.258+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:37.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:37.364+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:37.585 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:37.364+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:37.585 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:37 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:37.364+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:37.720 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:37.329+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:37.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:37.329+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:37.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:37.329+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:37.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:37.493+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:37.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:37.493+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:37.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:37.493+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:38.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:38 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:38.172+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:38.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:38 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:38.172+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:38.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:38 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:38.172+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:38.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:38 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:38.172+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:38.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:38.397+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:38.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:38.397+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:38.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:38.397+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:38.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:38.397+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:38.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:38.275+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:38.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:38.275+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:38.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:38.275+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:38.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:38 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:38.275+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:38.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:38 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:38.312+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:38.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:38 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:38.312+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:38.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:38 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:38.312+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:38.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:38 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:38.484+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:38.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:38 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:38.484+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:38.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:38 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:38.484+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:39.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:39.184+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:39.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:39.184+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:39.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:39.184+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:39.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:39 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:39.184+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:39.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:39.357+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:39.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:39.357+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:39.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:39.357+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:39.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:39.357+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:39.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:39.319+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:39.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:39.319+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:39.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:39.319+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:39.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:39.319+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:39.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:39.347+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:49.795506+0000 front 2023-08-11T15:57:49.795471+0000 (oldest deadline 2023-08-11T15:58:15.095590+0000) 2023-08-11T15:58:39.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:39.347+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.297186+0000 front 2023-08-11T15:58:00.297121+0000 (oldest deadline 2023-08-11T15:58:23.197205+0000) 2023-08-11T15:58:39.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:39.347+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:06.098338+0000 front 2023-08-11T15:58:06.098357+0000 (oldest deadline 2023-08-11T15:58:28.998278+0000) 2023-08-11T15:58:39.722 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7[146296]: 2023-08-11T15:58:39.347+0000 7f2906d8a700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.499470+0000 front 2023-08-11T15:58:12.499506+0000 (oldest deadline 2023-08-11T15:58:38.399569+0000) 2023-08-11T15:58:39.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:39.509+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:39.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:39.509+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:39.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:39.509+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:39.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:39 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:39.509+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:40.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:40.176+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:40.586 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:40.176+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:40.587 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:40.176+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:40.587 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:40 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:40.176+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:40.721 INFO:journalctl@ceph.osd.7.smithi134.stdout:Aug 11 15:58:40 smithi134 bash[162417]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-7 2023-08-11T15:58:40.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:40 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:40.462+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:40.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:40 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:40.462+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:40.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:40 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:40.462+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:40.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:40 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:40.462+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:40.833 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:40.359+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:40.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:40.359+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:40.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:40.359+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:40.834 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:40.359+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:40.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:40.336+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:40.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:40.336+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:40.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:40.336+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:40.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:40 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:40.336+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:41.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:41.163+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:41.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:41.163+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:41.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:41.163+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:41.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:41 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:41.163+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:41.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:41.375+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:41.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:41.375+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:41.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:41.375+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:41.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:41.375+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:41.585 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:41.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:41.585 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:41.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:41.585 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:41.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:41.585 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:41 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:41.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:41.625 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:41 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:41.414+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:41.625 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:41 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:41.414+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:41.626 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:41 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:41.414+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:41.626 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:41 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:41.414+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:41.882 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.7.service' 2023-08-11T15:58:41.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:41.964 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-08-11T15:58:41.964 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-08-11T15:58:41.964 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.8 2023-08-11T15:58:42.322 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 systemd[1]: Stopping Ceph osd.8 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:42.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:42 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:42.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:42.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:42 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:42.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:42.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:42 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:42.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:42.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:42 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:42.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:42.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:42 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:42.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:42.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:42.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:42.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:42.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:42.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:42.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:42.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:42.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:42.586 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:42.320+0000 7f548cda3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:42.586 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:42.320+0000 7f548cda3700 -1 osd.8 113 *** Got signal Terminated *** 2023-08-11T15:58:42.586 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:42.320+0000 7f548cda3700 -1 osd.8 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:42.587 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:42.337+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:42.587 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:42.337+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:42.588 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:42.337+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:42.588 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:42 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:42.337+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:42.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:42 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:42.453+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:42.725 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:42 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:42.453+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:42.725 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:42 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:42.453+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:42.726 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:42 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:42.453+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:43.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:43 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:43.232+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:43.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:43 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:43.232+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:43.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:43 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:43.232+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:43.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:43 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:43.232+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:43.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:43 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:43.232+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:43.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:43.326+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:43.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:43.326+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:43.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:43.326+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:43.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:43.326+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:43.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:43.380+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:43.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:43.380+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:43.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:43.380+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:43.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:43 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:43.380+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:43.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:43 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:43.423+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:43.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:43 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:43.423+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:43.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:43 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:43.423+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:43.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:43 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:43.423+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:44.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:44 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:44.228+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:44.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:44 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:44.228+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:44.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:44 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:44.228+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:44.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:44 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:44.228+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:44.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:44 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:44.228+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:44.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:44.277+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:44.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:44.277+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:44.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:44.277+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:44.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:44.277+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:44.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:44.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:44.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:44.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:44.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:44.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:44.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:44.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:44.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:44 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:44.421+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:44.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:44 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:44.421+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:44.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:44 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:44.421+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:44.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:44 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:44.421+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:45.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:45.208+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:45.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:45.208+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:45.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:45.208+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:45.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:45.208+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:45.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:45.208+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:45.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:45 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:45.208+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:45.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:45.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:45.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:45.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:45.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:45.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:45.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:45.325+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:45.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:45.367+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:45.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:45.367+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:45.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:45.367+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:45.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:45 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:45.367+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:45.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:45 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:45.418+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:45.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:45 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:45.418+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:45.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:45 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:45.418+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:45.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:45 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:45.418+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:46.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:46.195+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:46.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:46.195+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:46.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:46.195+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:46.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:46.195+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:46.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:46.195+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:46.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:46 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:46.195+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:46.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:46.305+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:51.870972+0000 front 2023-08-11T15:57:51.871179+0000 (oldest deadline 2023-08-11T15:58:15.271648+0000) 2023-08-11T15:58:46.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:46.305+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:59.972665+0000 front 2023-08-11T15:57:59.972628+0000 (oldest deadline 2023-08-11T15:58:23.973656+0000) 2023-08-11T15:58:46.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:46.305+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:03.974329+0000 front 2023-08-11T15:58:03.974240+0000 (oldest deadline 2023-08-11T15:58:29.274497+0000) 2023-08-11T15:58:46.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8[147344]: 2023-08-11T15:58:46.305+0000 7f5486393700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:16.276403+0000 front 2023-08-11T15:58:16.276197+0000 (oldest deadline 2023-08-11T15:58:37.376196+0000) 2023-08-11T15:58:46.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:46.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:46.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:46.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:46.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:46.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:46.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:46.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:46.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:46 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:46.339+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:46.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:46 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:46.463+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:46.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:46 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:46.463+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:46.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:46 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:46.463+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:46.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:46 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:46.463+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:47.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:47.161+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:47.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:47.161+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:47.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:47.161+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:47.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:47.161+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:47.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:47.161+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:47.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:47 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:47.161+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:47.833 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:47 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:47.378+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:47.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:47 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:47.378+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:47.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:47 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:47.378+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:47.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:47 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:47.378+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:47.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:47 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:47.378+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:47.971 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:47 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:47.473+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:47.971 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:47 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:47.473+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:47.972 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:47 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:47.473+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:47.972 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:47 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:47.473+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:48.333 INFO:journalctl@ceph.osd.8.smithi192.stdout:Aug 11 15:58:48 smithi192 bash[163399]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-8 2023-08-11T15:58:48.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:48.175+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:48.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:48.175+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:48.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:48.175+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:48.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:48.175+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:48.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:48.175+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:48.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:48 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:48.175+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:48.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:48 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:48.445+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:48.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:48 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:48.445+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:48.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:48 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:48.445+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:48.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:48 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:48.445+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:48.742 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:48 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:48.414+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:48.743 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:48 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:48.414+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:48.743 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:48 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:48.414+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:48.743 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:48 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:48.414+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:48.744 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:48 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:48.414+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:49.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.8.service' 2023-08-11T15:58:49.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:49.503 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-08-11T15:58:49.503 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-08-11T15:58:49.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.9 2023-08-11T15:58:49.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:49.193+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:49.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:49.193+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:49.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:49.193+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:49.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:49.193+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:49.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:49.193+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:49.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:49 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:49.193+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:49.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:49 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:49.409+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:49.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:49 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:49.409+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:49.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:49 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:49.409+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:49.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:49 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:49.409+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:49.833 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:49 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:49.371+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:49.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:49 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:49.371+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:49.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:49 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:49.371+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:49.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:49 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:49.371+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:49.835 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:49 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:49.371+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:50.052 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:49 smithi032 systemd[1]: Stopping Ceph osd.9 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:50.303 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.050+0000 7f615f876700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:50.303 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.050+0000 7f615f876700 -1 osd.9 113 *** Got signal Terminated *** 2023-08-11T15:58:50.304 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.050+0000 7f615f876700 -1 osd.9 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:50.304 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.179+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:50.304 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.179+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:50.304 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.179+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:50.305 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.179+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:50.305 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.179+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:50.305 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:50 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:50.179+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:50.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:50 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:50.379+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:50.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:50 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:50.379+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:50.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:50 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:50.379+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:50.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:50 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:50.379+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:50.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:50 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:50.379+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:50.833 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:50 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:50.395+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:50.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:50 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:50.395+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:50.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:50 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:50.395+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:50.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:50 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:50.395+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:50.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:50 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:50.395+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:51.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:51 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:51.189+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:51.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:51 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:51.189+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:51.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:51 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:51.189+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:51.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:51 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:51.189+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:51.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:51 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:51.189+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:51.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:51 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:51.189+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:51.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:51 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:51.378+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:51.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:51 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:51.378+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:51.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:51 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:51.378+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:51.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:51 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:51.378+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:51.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:51 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:51.378+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:51.833 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:51 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:51.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:51.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:51 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:51.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:51.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:51 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:51.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:51.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:51 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:51.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:51.834 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:51 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:51.361+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:52.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:52.168+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:52.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:52.168+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:52.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:52.168+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:52.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:52.168+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:52.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:52.168+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:52.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:52 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:52.168+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:52.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:52 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:52.312+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:52.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:52 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:52.312+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:52.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:52 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:52.312+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:52.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:52 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:52.312+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:52.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:52 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:52.312+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:52.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:52 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:52.383+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:52.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:52 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:52.383+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:52.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:52 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:52.383+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:52.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:52 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:52.383+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:52.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:52 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:52.383+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:53.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:53 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:53.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:53.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:53 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:53.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:53.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:53 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:53.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:53.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:53 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:53.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:53.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:53 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:53.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:53.555 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:53 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:53.165+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:53.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:53 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:53.270+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:53.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:53 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:53.270+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:53.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:53 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:53.270+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:53.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:53 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:53.270+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:53.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:53 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:53.270+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:53.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:53 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:53.334+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:53.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:53 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:53.334+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:53.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:53 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:53.334+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:53.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:53 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:53.334+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:53.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:53 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:53.334+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:54.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:54 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:54.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:42.832947+0000 front 2023-08-11T15:57:42.833506+0000 (oldest deadline 2023-08-11T15:58:07.433493+0000) 2023-08-11T15:58:54.553 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:54 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:54.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:52.735479+0000 front 2023-08-11T15:57:52.735453+0000 (oldest deadline 2023-08-11T15:58:16.835537+0000) 2023-08-11T15:58:54.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:54 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:54.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.238198+0000 front 2023-08-11T15:58:00.238145+0000 (oldest deadline 2023-08-11T15:58:21.938224+0000) 2023-08-11T15:58:54.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:54 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:54.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.740001+0000 front 2023-08-11T15:58:04.740027+0000 (oldest deadline 2023-08-11T15:58:30.540770+0000) 2023-08-11T15:58:54.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:54 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:54.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.143560+0000 front 2023-08-11T15:58:15.143669+0000 (oldest deadline 2023-08-11T15:58:42.144586+0000) 2023-08-11T15:58:54.554 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:54 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9[125756]: 2023-08-11T15:58:54.212+0000 7f615be6c700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:22.145220+0000 front 2023-08-11T15:58:22.145320+0000 (oldest deadline 2023-08-11T15:58:45.045304+0000) 2023-08-11T15:58:54.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:54 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:54.282+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:54.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:54 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:54.282+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:54.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:54 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:54.282+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:54.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:54 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:54.282+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:54.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:54 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:54.282+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:54.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:54 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:54.290+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:54.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:54 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:54.290+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:54.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:54 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:54.290+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:54.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:54 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:54.290+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:54.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:54 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:54.290+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:55.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:55.255+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:55.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:55.255+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:55.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:55.255+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:55.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:55.255+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:55.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:55 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:55.255+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:55.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:55 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:55.319+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:55.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:55 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:55.319+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:55.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:55 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:55.319+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:55.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:55 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:55.319+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:55.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:55 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:55.319+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:56.053 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 11 15:58:55 smithi032 bash[155899]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-9 2023-08-11T15:58:56.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:56 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:56.226+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:56.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:56 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:56.226+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:56.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:56 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:56.226+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:56.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:56 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:56.226+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:56.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:56 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:56.226+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:56.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:56 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:56.354+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:56.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:56 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:56.354+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:56.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:56 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:56.354+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:56.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:56 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:56.354+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:56.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:56 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:56.354+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:56.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:56 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:56.354+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:30.964400+0000 front 2023-08-11T15:58:30.964452+0000 (oldest deadline 2023-08-11T15:58:55.664204+0000) 2023-08-11T15:58:57.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:57.189+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:57.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:57.189+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:57.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:57.189+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:57.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:57.189+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:57.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:57 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:57.189+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:57.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:57 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:57.325+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:57.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:57 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:57.325+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:57.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:57 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:57.325+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:57.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:57 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:57.325+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:57.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:57 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:57.325+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:57.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:57 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:57.325+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:30.964400+0000 front 2023-08-11T15:58:30.964452+0000 (oldest deadline 2023-08-11T15:58:55.664204+0000) 2023-08-11T15:58:57.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.9.service' 2023-08-11T15:58:57.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:58:57.910 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-08-11T15:58:57.911 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-08-11T15:58:57.911 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.10 2023-08-11T15:58:58.220 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:57 smithi134 systemd[1]: Stopping Ceph osd.10 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:58:58.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:58 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:58.158+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:58.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:58 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:58.158+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:58.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:58 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:58.158+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:58.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:58 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:58.158+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:58.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:58 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:58.158+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:58.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:58 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:58.158+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:58:58.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.318+0000 7fdce89c3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:58:58.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.318+0000 7fdce89c3700 -1 osd.10 113 *** Got signal Terminated *** 2023-08-11T15:58:58.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.318+0000 7fdce89c3700 -1 osd.10 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:58:58.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.358+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:58.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.358+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:58.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.358+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:58.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.358+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:58.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.358+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:58.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:58 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:58.358+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:30.964400+0000 front 2023-08-11T15:58:30.964452+0000 (oldest deadline 2023-08-11T15:58:55.664204+0000) 2023-08-11T15:58:59.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:59.167+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:58:59.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:59.167+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:58:59.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:59.167+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:58:59.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:59.167+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:58:59.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:59.167+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:58:59.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:58:59 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:58:59.167+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:58:59.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:59.356+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:58:59.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:59.356+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:58:59.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:59.356+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:58:59.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:59.356+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:58:59.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:59.356+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:58:59.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:58:59 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:58:59.356+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:30.964400+0000 front 2023-08-11T15:58:30.964452+0000 (oldest deadline 2023-08-11T15:58:55.664204+0000) 2023-08-11T15:59:00.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:00.153+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:00.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:00.153+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:00.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:00.153+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:00.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:00.153+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:00.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:00.153+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:00.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:00 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:00.153+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:00.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:00.376+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:59:00.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:00.376+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:59:00.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:00.376+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:59:00.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:00.376+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:59:00.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:00.376+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:59:00.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:00 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:00.376+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:30.964400+0000 front 2023-08-11T15:58:30.964452+0000 (oldest deadline 2023-08-11T15:58:55.664204+0000) 2023-08-11T15:59:01.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:01.112+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:01.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:01.112+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:01.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:01.112+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:01.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:01.112+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:01.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:01.112+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:01.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:01.112+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:01.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:01 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:01.112+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:01.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:01.396+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:59:01.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:01.396+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:59:01.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:01.396+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:59:01.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:01.396+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:59:01.723 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:01.396+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:59:01.724 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:01 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:01.396+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:30.964400+0000 front 2023-08-11T15:58:30.964452+0000 (oldest deadline 2023-08-11T15:58:55.664204+0000) 2023-08-11T15:59:02.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:02.147+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:02.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:02.147+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:02.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:02.147+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:02.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:02.147+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:02.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:02.147+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:02.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:02.147+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:02.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:02 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:02.147+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:02.720 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:02.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-08-11T15:57:47.757090+0000 front 2023-08-11T15:57:47.757064+0000 (oldest deadline 2023-08-11T15:58:14.158011+0000) 2023-08-11T15:59:02.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:02.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:58:00.059098+0000 front 2023-08-11T15:58:00.059076+0000 (oldest deadline 2023-08-11T15:58:22.359079+0000) 2023-08-11T15:59:02.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:02.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.060260+0000 front 2023-08-11T15:58:04.060323+0000 (oldest deadline 2023-08-11T15:58:29.960333+0000) 2023-08-11T15:59:02.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:02.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:12.861381+0000 front 2023-08-11T15:58:12.861355+0000 (oldest deadline 2023-08-11T15:58:38.761513+0000) 2023-08-11T15:59:02.721 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:02.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:19.862471+0000 front 2023-08-11T15:58:19.862406+0000 (oldest deadline 2023-08-11T15:58:49.863214+0000) 2023-08-11T15:59:02.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:02.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:30.964400+0000 front 2023-08-11T15:58:30.964452+0000 (oldest deadline 2023-08-11T15:58:55.664204+0000) 2023-08-11T15:59:02.722 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:02 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10[151868]: 2023-08-11T15:59:02.440+0000 7fdce37b6700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:36.765087+0000 front 2023-08-11T15:58:36.765138+0000 (oldest deadline 2023-08-11T15:59:02.065298+0000) 2023-08-11T15:59:03.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:03.172+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:03.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:03.172+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:03.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:03.172+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:03.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:03.172+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:03.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:03.172+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:03.586 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:03.172+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:03.586 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:03 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:03.172+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:04.470 INFO:journalctl@ceph.osd.10.smithi134.stdout:Aug 11 15:59:04 smithi134 bash[162687]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-10 2023-08-11T15:59:04.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:04.188+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:04.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:04.188+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:04.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:04.188+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:04.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:04.188+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:04.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:04.188+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:04.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:04.188+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:04.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:04 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:04.188+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:05.459 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.10.service' 2023-08-11T15:59:05.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:05.500 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-08-11T15:59:05.500 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-08-11T15:59:05.501 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.11 2023-08-11T15:59:05.532 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.210+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:05.532 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.210+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:05.532 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.210+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:05.533 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.210+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:05.533 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.210+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:05.533 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.210+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:05.533 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.210+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:05.833 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 systemd[1]: Stopping Ceph osd.11 for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:06.203 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.859+0000 7f839703b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:06.203 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.859+0000 7f839703b700 -1 osd.11 113 *** Got signal Terminated *** 2023-08-11T15:59:06.203 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:05 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:05.859+0000 7f839703b700 -1 osd.11 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T15:59:06.204 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:06 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:06.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:06.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:06 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:06.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:06.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:06 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:06.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:06.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:06 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:06.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:06.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:06 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:06.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:06.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:06 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:06.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:06.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:06 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:06.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:07.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:07.160+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:07.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:07.160+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:07.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:07.160+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:07.586 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:07.160+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:07.586 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:07.160+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:07.587 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:07.160+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:07.587 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:07 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:07.160+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:08.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:08.140+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:08.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:08.140+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:08.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:08.140+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:08.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:08.140+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:08.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:08.140+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:08.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:08.140+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:08.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:08 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:08.140+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:09.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:09.183+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:09.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:09.183+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:09.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:09.183+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:09.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:09.183+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:09.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:09.183+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:09.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:09.183+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:09.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:09 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:09.183+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:10.583 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-08-11T15:57:45.099546+0000 front 2023-08-11T15:57:45.099441+0000 (oldest deadline 2023-08-11T15:58:10.399398+0000) 2023-08-11T15:59:10.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-08-11T15:57:58.601170+0000 front 2023-08-11T15:57:58.601142+0000 (oldest deadline 2023-08-11T15:58:21.501324+0000) 2023-08-11T15:59:10.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-08-11T15:58:04.902944+0000 front 2023-08-11T15:58:04.902984+0000 (oldest deadline 2023-08-11T15:58:29.003009+0000) 2023-08-11T15:59:10.584 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-08-11T15:58:15.404667+0000 front 2023-08-11T15:58:15.404684+0000 (oldest deadline 2023-08-11T15:58:37.704596+0000) 2023-08-11T15:59:10.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-08-11T15:58:21.706114+0000 front 2023-08-11T15:58:21.706212+0000 (oldest deadline 2023-08-11T15:58:45.706695+0000) 2023-08-11T15:59:10.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-08-11T15:58:29.708627+0000 front 2023-08-11T15:58:29.708484+0000 (oldest deadline 2023-08-11T15:58:57.309076+0000) 2023-08-11T15:59:10.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-08-11T15:58:37.309846+0000 front 2023-08-11T15:58:37.309794+0000 (oldest deadline 2023-08-11T15:59:00.809664+0000) 2023-08-11T15:59:10.585 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:10 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11[152841]: 2023-08-11T15:59:10.201+0000 7f839162d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2023-08-11T15:58:46.511725+0000 front 2023-08-11T15:58:46.511742+0000 (oldest deadline 2023-08-11T15:59:09.411824+0000) 2023-08-11T15:59:11.541 INFO:journalctl@ceph.osd.11.smithi192.stdout:Aug 11 15:59:11 smithi192 bash[163668]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-osd-11 2023-08-11T15:59:12.803 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@osd.11.service' 2023-08-11T15:59:12.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:12.884 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-08-11T15:59:12.884 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-08-11T15:59:12.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.a 2023-08-11T15:59:13.303 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:59:12 smithi032 systemd[1]: Stopping Ceph mds.a for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:13.803 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:59:13 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: 2023-08-11T15:59:13.385+0000 7f7334443700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:13.803 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:59:13 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a[131139]: 2023-08-11T15:59:13.386+0000 7f7334443700 -1 mds.a *** got signal Terminated *** 2023-08-11T15:59:15.303 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 11 15:59:14 smithi032 bash[156172]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-a 2023-08-11T15:59:15.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.a.service' 2023-08-11T15:59:15.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:15.435 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-08-11T15:59:15.436 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-08-11T15:59:15.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.d 2023-08-11T15:59:15.803 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:59:15 smithi032 systemd[1]: Stopping Ceph mds.d for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:16.303 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:59:15 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-d[130181]: 2023-08-11T15:59:15.943+0000 7fbb8e0f4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:16.303 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:59:15 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-d[130181]: 2023-08-11T15:59:15.943+0000 7fbb8e0f4700 -1 mds.d *** got signal Terminated *** 2023-08-11T15:59:17.413 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 11 15:59:17 smithi032 bash[156314]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-d 2023-08-11T15:59:17.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.d.service' 2023-08-11T15:59:17.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:17.799 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-08-11T15:59:17.799 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-08-11T15:59:17.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.g 2023-08-11T15:59:18.053 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:59:17 smithi032 systemd[1]: Stopping Ceph mds.g for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:18.803 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:59:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-g[130751]: 2023-08-11T15:59:18.343+0000 7f49cfe4b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:18.804 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:59:18 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-g[130751]: 2023-08-11T15:59:18.343+0000 7f49cfe4b700 -1 mds.g *** got signal Terminated *** 2023-08-11T15:59:22.802 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 11 15:59:22 smithi032 bash[156460]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-g 2023-08-11T15:59:23.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.g.service' 2023-08-11T15:59:23.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:23.195 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-08-11T15:59:23.196 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-08-11T15:59:23.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.j 2023-08-11T15:59:23.553 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:59:23 smithi032 systemd[1]: Stopping Ceph mds.j for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:24.053 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:59:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-j[129796]: 2023-08-11T15:59:23.735+0000 7f1dfc6f1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:24.053 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:59:23 smithi032 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-j[129796]: 2023-08-11T15:59:23.735+0000 7f1dfc6f1700 -1 mds.j *** got signal Terminated *** 2023-08-11T15:59:26.383 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:59:26 smithi032 bash[156607]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-j 2023-08-11T15:59:26.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.j.service' 2023-08-11T15:59:26.640 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:59:26 smithi032 systemd[1]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.j.service: Succeeded. 2023-08-11T15:59:26.641 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 11 15:59:26 smithi032 systemd[1]: Stopped Ceph mds.j for cd36a838-385c-11ee-9b37-001a4aab830c. 2023-08-11T15:59:26.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:26.663 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-08-11T15:59:26.663 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-08-11T15:59:26.664 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.b 2023-08-11T15:59:26.970 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:59:26 smithi134 systemd[1]: Stopping Ceph mds.b for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:27.470 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:59:27 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b[155523]: 2023-08-11T15:59:27.017+0000 7f8f3cb48700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:27.471 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:59:27 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b[155523]: 2023-08-11T15:59:27.017+0000 7f8f3cb48700 -1 mds.b *** got signal Terminated *** 2023-08-11T15:59:28.720 INFO:journalctl@ceph.mds.b.smithi134.stdout:Aug 11 15:59:28 smithi134 bash[162956]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-b 2023-08-11T15:59:28.765 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.b.service' 2023-08-11T15:59:28.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:28.801 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-08-11T15:59:28.801 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-08-11T15:59:28.802 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.e 2023-08-11T15:59:29.167 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:59:28 smithi134 systemd[1]: Stopping Ceph mds.e for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:29.470 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:59:29 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-e[154839]: 2023-08-11T15:59:29.165+0000 7f6bc12d8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:29.471 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:59:29 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-e[154839]: 2023-08-11T15:59:29.165+0000 7f6bc12d8700 -1 mds.e *** got signal Terminated *** 2023-08-11T15:59:31.540 INFO:journalctl@ceph.mds.e.smithi134.stdout:Aug 11 15:59:31 smithi134 bash[163097]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-e 2023-08-11T15:59:31.729 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.e.service' 2023-08-11T15:59:31.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:31.763 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-08-11T15:59:31.764 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-08-11T15:59:31.764 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.h 2023-08-11T15:59:32.134 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:59:31 smithi134 systemd[1]: Stopping Ceph mds.h for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:32.470 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:59:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: 2023-08-11T15:59:32.132+0000 7f544e193700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:32.471 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:59:32 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h[155182]: 2023-08-11T15:59:32.132+0000 7f544e193700 -1 mds.h *** got signal Terminated *** 2023-08-11T15:59:36.601 INFO:journalctl@ceph.mds.h.smithi134.stdout:Aug 11 15:59:36 smithi134 bash[163238]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-h 2023-08-11T15:59:36.606 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.h.service' 2023-08-11T15:59:36.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:36.640 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-08-11T15:59:36.640 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-08-11T15:59:36.641 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.k 2023-08-11T15:59:36.970 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:59:36 smithi134 systemd[1]: Stopping Ceph mds.k for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:37.470 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:59:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-k[154502]: 2023-08-11T15:59:37.005+0000 7f0875bcb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:37.471 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:59:37 smithi134 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-k[154502]: 2023-08-11T15:59:37.005+0000 7f0875bcb700 -1 mds.k *** got signal Terminated *** 2023-08-11T15:59:39.440 INFO:journalctl@ceph.mds.k.smithi134.stdout:Aug 11 15:59:39 smithi134 bash[163395]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-k 2023-08-11T15:59:39.587 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.k.service' 2023-08-11T15:59:39.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:39.624 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-08-11T15:59:39.624 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-08-11T15:59:39.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.c 2023-08-11T15:59:39.988 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:59:39 smithi192 systemd[1]: Stopping Ceph mds.c for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:40.333 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:59:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-c[156425]: 2023-08-11T15:59:39.986+0000 7f6855959700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:40.334 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:59:39 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-c[156425]: 2023-08-11T15:59:39.986+0000 7f6855959700 -1 mds.c *** got signal Terminated *** 2023-08-11T15:59:43.854 INFO:journalctl@ceph.mds.c.smithi192.stdout:Aug 11 15:59:43 smithi192 bash[163933]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-c 2023-08-11T15:59:44.042 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.c.service' 2023-08-11T15:59:44.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:44.077 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-08-11T15:59:44.078 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-08-11T15:59:44.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.f 2023-08-11T15:59:44.333 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:59:44 smithi192 systemd[1]: Stopping Ceph mds.f for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:44.833 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:59:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-f[156085]: 2023-08-11T15:59:44.453+0000 7f14e3954700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:44.834 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:59:44 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-f[156085]: 2023-08-11T15:59:44.453+0000 7f14e3954700 -1 mds.f *** got signal Terminated *** 2023-08-11T15:59:49.040 INFO:journalctl@ceph.mds.f.smithi192.stdout:Aug 11 15:59:48 smithi192 bash[164073]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-f 2023-08-11T15:59:49.046 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.f.service' 2023-08-11T15:59:49.080 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:49.081 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-08-11T15:59:49.082 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-08-11T15:59:49.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.i 2023-08-11T15:59:49.333 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:59:49 smithi192 systemd[1]: Stopping Ceph mds.i for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:49.833 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:59:49 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-i[156762]: 2023-08-11T15:59:49.451+0000 7f9d7ae02700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:49.834 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:59:49 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-i[156762]: 2023-08-11T15:59:49.451+0000 7f9d7ae02700 -1 mds.i *** got signal Terminated *** 2023-08-11T15:59:52.333 INFO:journalctl@ceph.mds.i.smithi192.stdout:Aug 11 15:59:52 smithi192 bash[164219]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-i 2023-08-11T15:59:52.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.i.service' 2023-08-11T15:59:52.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:52.412 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-08-11T15:59:52.413 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-08-11T15:59:52.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.l 2023-08-11T15:59:52.803 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:59:52 smithi192 systemd[1]: Stopping Ceph mds.l for cd36a838-385c-11ee-9b37-001a4aab830c... 2023-08-11T15:59:53.083 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:59:52 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-l[155736]: 2023-08-11T15:59:52.801+0000 7f4034538700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T15:59:53.084 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:59:52 smithi192 ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-l[155736]: 2023-08-11T15:59:52.801+0000 7f4034538700 -1 mds.l *** got signal Terminated *** 2023-08-11T15:59:57.293 INFO:journalctl@ceph.mds.l.smithi192.stdout:Aug 11 15:59:56 smithi192 bash[164364]: ceph-cd36a838-385c-11ee-9b37-001a4aab830c-mds-l 2023-08-11T15:59:57.299 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cd36a838-385c-11ee-9b37-001a4aab830c@mds.l.service' 2023-08-11T15:59:57.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T15:59:57.335 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-08-11T15:59:57.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cd36a838-385c-11ee-9b37-001a4aab830c --force --keep-logs 2023-08-11T15:59:57.738 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T16:00:01.116 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cd36a838-385c-11ee-9b37-001a4aab830c --force --keep-logs 2023-08-11T16:00:01.452 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T16:00:03.097 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cd36a838-385c-11ee-9b37-001a4aab830c --force --keep-logs 2023-08-11T16:00:03.407 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T16:00:04.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T16:00:05.075 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T16:00:05.112 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T16:00:05.143 INFO:tasks.cephadm:Archiving crash dumps... 2023-08-11T16:00:05.147 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi032/crash 2023-08-11T16:00:05.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash -- . 2023-08-11T16:00:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:tar: /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-11T16:00:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:tar: Error is not recoverable: exiting now 2023-08-11T16:00:05.232 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi134/crash 2023-08-11T16:00:05.233 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash -- . 2023-08-11T16:00:05.283 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-11T16:00:05.284 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2023-08-11T16:00:05.287 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi192/crash 2023-08-11T16:00:05.290 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash -- . 2023-08-11T16:00:05.322 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-11T16:00:05.323 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2023-08-11T16:00:05.323 INFO:tasks.cephadm:Checking cluster log for badness... 2023-08-11T16:00:05.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-11T16:00:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:grep: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph.log: No such file or directory 2023-08-11T16:00:05.407 INFO:tasks.cephadm:Compressing logs... 2023-08-11T16:00:05.408 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T16:00:05.413 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T16:00:05.416 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T16:00:05.443 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.5.log 2023-08-11T16:00:05.443 INFO:teuthology.orchestra.run.smithi192.stderr:: No such file or directory 2023-08-11T16:00:05.444 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.2.log 2023-08-11T16:00:05.444 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.11.log 2023-08-11T16:00:05.445 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.5.log: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.z.log 2023-08-11T16:00:05.446 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.8.log 2023-08-11T16:00:05.446 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.f.log 2023-08-11T16:00:05.446 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.c.log 2023-08-11T16:00:05.449 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.f.log: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.c.log: 94.5% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.f.log.gz 2023-08-11T16:00:05.450 INFO:teuthology.orchestra.run.smithi192.stderr: 94.5% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.c.log.gz 2023-08-11T16:00:05.450 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.i.log 2023-08-11T16:00:05.454 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.c.log 2023-08-11T16:00:05.455 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.i.log: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.c.log: 91.9% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.z.log.gz 2023-08-11T16:00:05.455 INFO:teuthology.orchestra.run.smithi192.stderr: 94.4% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.i.log.gz 2023-08-11T16:00:05.456 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log 2023-08-11T16:00:05.456 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.l.log 2023-08-11T16:00:05.461 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-11T16:00:05.461 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-11T16:00:05.461 INFO:teuthology.orchestra.run.smithi192.stderr: 94.5% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.l.log.gz 2023-08-11T16:00:05.462 INFO:teuthology.orchestra.run.smithi192.stderr: 91.2% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log.gz 2023-08-11T16:00:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/var/log/rbd-target-api’gzip -5 : No such file or directory--verbose 2023-08-11T16:00:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:-- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.0.log 2023-08-11T16:00:05.477 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.g.log 2023-08-11T16:00:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.a.log 2023-08-11T16:00:05.479 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.6.log 2023-08-11T16:00:05.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.3.log 2023-08-11T16:00:05.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.d.log 2023-08-11T16:00:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.3.log: 94.4% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.g.log.gz 2023-08-11T16:00:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.x.log 2023-08-11T16:00:05.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.9.log 2023-08-11T16:00:05.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.x.log: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log 2023-08-11T16:00:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.d.log.gz 2023-08-11T16:00:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.j.log 2023-08-11T16:00:05.499 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.j.log: 91.1% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log.gz 2023-08-11T16:00:05.499 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.a.log 2023-08-11T16:00:05.499 INFO:teuthology.orchestra.run.smithi032.stderr: 94.5% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.j.log.gz 2023-08-11T16:00:05.500 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-11T16:00:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-08-11T16:00:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.10.log 2023-08-11T16:00:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.h.log 2023-08-11T16:00:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.e.log 2023-08-11T16:00:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.10.log: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.4.log 2023-08-11T16:00:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.1.log 2023-08-11T16:00:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.k.log 2023-08-11T16:00:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log 2023-08-11T16:00:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.7.log 2023-08-11T16:00:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.b.log 2023-08-11T16:00:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.b.log 2023-08-11T16:00:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.7.log: /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.y.log 2023-08-11T16:00:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-11T16:00:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.y.log: /var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.e.log.gz 2023-08-11T16:00:05.517 INFO:teuthology.orchestra.run.smithi134.stderr: 94.5% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.k.log.gz 2023-08-11T16:00:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.a.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-08-11T16:00:05.519 INFO:teuthology.orchestra.run.smithi032.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-11T16:00:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.log: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-08-11T16:00:05.524 INFO:teuthology.orchestra.run.smithi134.stderr: 91.8% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.y.log.gz 2023-08-11T16:00:05.539 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-11T16:00:05.821 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mgr.x.log.gz 2023-08-11T16:00:05.850 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.b.log.gz 2023-08-11T16:00:06.433 INFO:teuthology.orchestra.run.smithi192.stderr: 92.8% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.c.log.gz 2023-08-11T16:00:06.775 INFO:teuthology.orchestra.run.smithi032.stderr: 90.5% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mon.a.log.gz 2023-08-11T16:00:07.340 INFO:teuthology.orchestra.run.smithi134.stderr: 91.2% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-volume.log.gz 2023-08-11T16:00:09.347 INFO:teuthology.orchestra.run.smithi134.stderr: 95.4% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.h.log.gz 2023-08-11T16:00:10.742 INFO:teuthology.orchestra.run.smithi032.stderr: 95.5% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.a.log.gz 2023-08-11T16:00:21.907 INFO:teuthology.orchestra.run.smithi192.stderr: 91.7% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.11.log.gz 2023-08-11T16:00:22.777 INFO:teuthology.orchestra.run.smithi192.stderr: 91.7% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.8.log.gz 2023-08-11T16:00:22.836 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.10.log.gz 2023-08-11T16:00:25.690 INFO:teuthology.orchestra.run.smithi032.stderr: 92.0% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.0.log.gz 2023-08-11T16:00:27.090 INFO:teuthology.orchestra.run.smithi192.stderr: 91.8% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.2.log.gz 2023-08-11T16:00:28.230 INFO:teuthology.orchestra.run.smithi032.stderr: 91.7% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.6.log.gz 2023-08-11T16:00:29.297 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.1.log.gz 2023-08-11T16:00:29.851 INFO:teuthology.orchestra.run.smithi032.stderr: 91.7% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.3.log.gz 2023-08-11T16:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr: 91.7% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.9.log.gz 2023-08-11T16:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-08-11T16:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m24.918s 2023-08-11T16:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:user 1m31.882s 2023-08-11T16:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m7.308s 2023-08-11T16:00:30.925 INFO:teuthology.orchestra.run.smithi134.stderr: 91.8% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.4.log.gz 2023-08-11T16:00:32.264 INFO:teuthology.orchestra.run.smithi192.stderr: 91.4% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.5.log.gz 2023-08-11T16:00:32.266 INFO:teuthology.orchestra.run.smithi192.stderr: 2023-08-11T16:00:32.266 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m26.834s 2023-08-11T16:00:32.266 INFO:teuthology.orchestra.run.smithi192.stderr:user 1m20.532s 2023-08-11T16:00:32.267 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m2.635s 2023-08-11T16:00:33.853 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-osd.7.log.gz 2023-08-11T16:02:19.268 INFO:teuthology.orchestra.run.smithi134.stderr: 93.6% -- replaced with /var/log/ceph/cd36a838-385c-11ee-9b37-001a4aab830c/ceph-mds.b.log.gz 2023-08-11T16:02:19.270 INFO:teuthology.orchestra.run.smithi134.stderr: 2023-08-11T16:02:19.271 INFO:teuthology.orchestra.run.smithi134.stderr:real 2m13.839s 2023-08-11T16:02:19.271 INFO:teuthology.orchestra.run.smithi134.stderr:user 3m36.807s 2023-08-11T16:02:19.271 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m7.341s 2023-08-11T16:02:19.272 INFO:tasks.cephadm:Archiving logs... 2023-08-11T16:02:19.272 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi032/log 2023-08-11T16:02:19.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T16:02:30.680 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi134/log 2023-08-11T16:02:30.681 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T16:02:57.407 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi192/log 2023-08-11T16:02:57.408 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T16:03:08.346 INFO:tasks.cephadm:Removing cluster... 2023-08-11T16:03:08.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cd36a838-385c-11ee-9b37-001a4aab830c --force 2023-08-11T16:03:08.747 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T16:03:09.827 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cd36a838-385c-11ee-9b37-001a4aab830c --force 2023-08-11T16:03:10.125 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T16:03:10.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cd36a838-385c-11ee-9b37-001a4aab830c --force 2023-08-11T16:03:11.180 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: cd36a838-385c-11ee-9b37-001a4aab830c 2023-08-11T16:03:11.770 INFO:tasks.cephadm:Removing cephadm ... 2023-08-11T16:03:11.771 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-11T16:03:11.826 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-11T16:03:11.864 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-11T16:03:11.888 INFO:tasks.cephadm:Teardown complete 2023-08-11T16:03:11.888 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-11T16:03:11.911 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-11T16:03:11.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T16:03:11.917 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T16:03:11.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T16:03:12.058 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-11T16:03:12.076 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-11T16:03:12.077 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T16:03:12.082 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T16:03:12.084 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T16:03:12.100 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-08-11T16:03:12.101 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2023-08-11T16:03:12.105 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T16:03:12.106 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-08-11T16:03:12.106 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 +877us[ +869us] +/- 78ms 2023-08-11T16:03:12.106 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 -906us[ -913us] +/- 55ms 2023-08-11T16:03:12.106 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +399us[ +399us] +/- 74ms 2023-08-11T16:03:12.107 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T16:03:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-08-11T16:03:12.128 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T16:03:12.129 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-08-11T16:03:12.129 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 +905us[ +922us] +/- 78ms 2023-08-11T16:03:12.129 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -875us[ -859us] +/- 55ms 2023-08-11T16:03:12.129 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +419us[ +436us] +/- 74ms 2023-08-11T16:03:12.129 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T16:03:12.131 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T16:03:12.131 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-08-11T16:03:12.131 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +894us[ +894us] +/- 78ms 2023-08-11T16:03:12.131 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 -891us[ -875us] +/- 55ms 2023-08-11T16:03:12.132 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +412us[ +412us] +/- 74ms 2023-08-11T16:03:12.132 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T16:03:12.133 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-11T16:03:12.144 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-11T16:03:12.146 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-11T16:03:12.169 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T16:03:12.353 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T16:03:12.480 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T16:03:12.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T16:03:12.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T16:03:12.653 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T16:03:12.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T16:03:12.690 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T16:03:12.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T16:03:12.723 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-11T16:03:12.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-11T16:03:13.040 INFO:teuthology.task.internal:Duration was 2641.943658 seconds 2023-08-11T16:03:13.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-11T16:03:13.144 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-11T16:03:13.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T16:03:13.150 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T16:03:13.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T16:03:13.201 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T16:03:13.244 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T16:03:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T16:03:13.595 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-11T16:03:13.596 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2023-08-11T16:03:13.596 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T16:03:13.655 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2023-08-11T16:03:13.656 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T16:03:13.686 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2023-08-11T16:03:13.686 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T16:03:13.708 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-11T16:03:13.708 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T16:03:13.713 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T16:03:13.728 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T16:03:13.809 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-11T16:03:13.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T16:03:13.813 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T16:03:13.815 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T16:03:14.011 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-11T16:03:14.026 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-11T16:03:14.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T16:03:14.097 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T16:03:14.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T16:03:14.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-11T16:03:14.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T16:03:14.197 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T16:03:14.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T16:03:14.236 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2023-08-11T16:03:14.247 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2023-08-11T16:03:14.261 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2023-08-11T16:03:14.298 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T16:03:14.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T16:03:14.340 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T16:03:14.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T16:03:14.356 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T16:03:14.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T16:03:14.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-11T16:03:14.384 INFO:teuthology.task.internal:Transferring archived files... 2023-08-11T16:03:14.385 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi032 2023-08-11T16:03:14.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T16:03:14.467 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi134 2023-08-11T16:03:14.468 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T16:03:14.505 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050/remote/smithi192 2023-08-11T16:03:14.506 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T16:03:14.542 INFO:teuthology.task.internal:Removing archive directory... 2023-08-11T16:03:14.543 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T16:03:14.548 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T16:03:14.550 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T16:03:14.594 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-11T16:03:14.611 INFO:teuthology.task.internal:Not uploading archives. 2023-08-11T16:03:14.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-11T16:03:14.625 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-11T16:03:14.626 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T16:03:14.631 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T16:03:14.634 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T16:03:14.648 INFO:teuthology.orchestra.run.smithi134.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 16:03 /home/ubuntu/cephtest 2023-08-11T16:03:14.650 INFO:teuthology.orchestra.run.smithi192.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 16:03 /home/ubuntu/cephtest 2023-08-11T16:03:14.670 INFO:teuthology.orchestra.run.smithi032.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 16:03 /home/ubuntu/cephtest 2023-08-11T16:03:14.673 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-11T16:03:14.686 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-11T16:03:14.716 INFO:teuthology.nuke:Checking targets against current locks 2023-08-11T16:03:14.745 DEBUG:teuthology.nuke:shortname: smithi032 2023-08-11T16:03:14.745 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T16:03:14.769 DEBUG:teuthology.nuke:shortname: smithi134 2023-08-11T16:03:14.769 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T16:03:14.773 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:10:31.166259', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-11T16:03:14.791 DEBUG:teuthology.nuke:shortname: smithi192 2023-08-11T16:03:14.792 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T16:03:14.796 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:10:31.164699', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-11T16:03:14.826 INFO:teuthology.orchestra.console.smithi032:Power off 2023-08-11T16:03:14.827 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T16:03:14.842 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7366050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 15:10:31.162949', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-11T16:03:14.852 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-08-11T16:03:14.852 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:14.877 INFO:teuthology.orchestra.console.smithi134:Power off 2023-08-11T16:03:14.878 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T16:03:14.895 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-08-11T16:03:14.903 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2023-08-11T16:03:14.904 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:14.924 INFO:teuthology.orchestra.console.smithi192:Power off 2023-08-11T16:03:14.925 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T16:03:14.935 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2023-08-11T16:03:14.948 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2023-08-11T16:03:14.948 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:14.975 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-08-11T16:03:18.896 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:18.937 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:18.977 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:19.009 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-08-11T16:03:19.051 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2023-08-11T16:03:19.091 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-08-11T16:03:23.012 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:23.052 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:23.092 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:23.124 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-08-11T16:03:23.164 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2023-08-11T16:03:23.204 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-08-11T16:03:23.226 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-08-11T16:03:23.266 INFO:teuthology.orchestra.console.smithi134:Power off completed 2023-08-11T16:03:27.208 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T16:03:27.319 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-08-11T16:03:27.420 INFO:teuthology.orchestra.console.smithi192:Power off completed 2023-08-11T16:03:27.526 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 2641.9436581134796 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92684807f016ce045ac66e37fdde6f8c47857bcc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=40ccea2273d247b0812d9d3cc46113ea status: fail success: false 2023-08-11T16:03:27.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T16:03:27.608 INFO:teuthology.run:FAIL